To prevent spam users, you can only post on this forum after registration, which is by invitation. If you want to post on the forum, please send me a mail (h DOT m DOT w DOT verbeek AT tue DOT nl) and I'll send you an invitation in return for an account.

How to compare process models?

fordanic
edited April 2015 in Process Mining

I'm interested in being able to compare a number of different process models but I'm not sure how to do it. I came across the paper: Process Equivalence: Comparing Two Process Models Based on Observed Behavior, but as I understand this paper only presents a method for comparing two models using a common set of event logs. However, in my case I'm interested in comparing two (or or more) process models with separate sets of event logs.

Any suggestions?

Answers

  • Dear Fordanic,


    To my knowledge to 'ready to go' plug-ins are available for this in ProM, although work is undergoing in comparative process mining. The main obstacle is however that the layout/visualization of the models needs to be synchronized in order to be able to quickly spot commonalities and differences.

    Joos Buijs

    Senior Data Scientist and process mining expert at APG (Dutch pension fund executor).
    Previously Assistant Professor in Process Mining at Eindhoven University of Technology
  • I'm assuming your first sentence should be "To my knowledge no 'ready to go' plug-ins are available for this in ProM", or?

    I guess the ongoing work you're referring to is related to the process cubes and for example the paper "Comparative Process Mining in Education: An Approach Based on Process Cubes". Haven't read the paper yet (just the abstract) but does it suggest a quantitative way of comparing process models or is it merely talking about a qualitative/visual assessment of differences (as your reply suggests)?

  • Hmh, wouldn't your Comparison Framework plugin be useful for this purpose?

  • Dear Fordanic,

    You could indeed use this framework, and add comparators to it that somehow synchronise layout. At the moment, it is a rather basic framework with some very specific implementations that I needed.

    If you want, you can extend it (you can rather easily add statistics on the log, model or model+log level). Please keep me up to date of your additions :)

    Joos Buijs

    Senior Data Scientist and process mining expert at APG (Dutch pension fund executor).
    Previously Assistant Professor in Process Mining at Eindhoven University of Technology
  • Hi Fordanic,

    I just saw this fresh blog post coming by, maybe something for you to investigate further?

    http://bpmve.blogspot.com.au/2015/04/paper-process-visualization-techniques.html

    Joos Buijs

    Senior Data Scientist and process mining expert at APG (Dutch pension fund executor).
    Previously Assistant Professor in Process Mining at Eindhoven University of Technology
  • Hi Fordanic,

    You're in luck, through a question of a colleague I remembered this Master thesis that was defended in our group about a month ago on exactly your problem :)

    See http://alexandria.tue.nl/extra1/afstversl/wsk-i/Mourik_2015.pdf

    I hope the plug-ins work, although they are not available through the ProM package manager, you can check out the code from SVN and run it from Eclipse (there are guides on how to do that on https://svn.win.tue.nl/trac/prom/wiki/Contribute and other pages)

    Joos Buijs

    Senior Data Scientist and process mining expert at APG (Dutch pension fund executor).
    Previously Assistant Professor in Process Mining at Eindhoven University of Technology
  • Hmh, correct me if I'm wrong, but both the referenced master's thesis and the paper in the blogpost refer to techniques for visual assessment of differences between process models, i.e. the comparison is qualitative in nature. Are you aware of any attempts of deriving more quantitaive approaches?

    For example, something similar to what is discussed in "Similarity of Business Process Models: Metrics and Evaluation", where label similarity, strucutural similarity and behavioral similarity are discusssed. Also, I'm thinking it should be possible to somehow define a metric based upon estimated trace fitness values between a set of derived process models and their corresponding event logs. If not being able to define a distance between process models but at least being able to rank a set of process models in terms of relative similarity.

  • Hi Fordanic,

    You're right, my pointers were mainly visually/qualitative.

    The paper you refer to would be my best bet indeed of more quantitative approaches for process model comparison. Your proposed solution would fall in the behavioral similarity class, and could already be discussed in the paper. There are some cave-ats such as how to simulate loops etc., but the general idea is nice.

    Joos Buijs

    Senior Data Scientist and process mining expert at APG (Dutch pension fund executor).
    Previously Assistant Professor in Process Mining at Eindhoven University of Technology
  • aronlindberg
    edited November 2016
    @JBuijs & @fordanic: Has there been any progress in this area? I'm creating multiple process models for the same dataset, and I would like to be able to compare which fits best. Is there a way to test this statistically, i.e. the way you would compare two regression models or SEM models using e.g. a Chi2-test?
  • Hi Fordanic, 
    I need to compare different process models and prefer quantitative metrics. have you found a way or implemented a plugin for this purpose ? 

    thanks 


  • Dear Amirah,
    There is a plug in in ProM named Compare Causal Nets (GED) which gives a GED distance between two causal nets.
Sign In or Register to comment.