To prevent spam users, you can only post on this forum after registration, which is by invitation. If you want to post on the forum, please send me a mail (h DOT m DOT w DOT verbeek AT tue DOT nl) and I'll send you an invitation in return for an account.

Convert CSV to XES using command line

hypermau
hypermau
in - Usage
Hello, I've use the import tool to convert CSV to XES using the UI.

A couple of question:

1. do you know if there is alredy a script to convert CSV to XES using  command line?
2. where I can find the source code of the converter ? I couldn't find it in the /packages dir.

Thank you

Comments

  • fmannhardt
    fmannhardt
    Hi, I created a command line version of the CSV importer. It is located here:
    https://svn.win.tue.nl/repos/prom/Packages/Log/Trunk/src/org/processmining/log/csvimport/CSVConversionCLI.java
    The rest of the source code is also there.

    Unfortunately, there are some links to the ProM framework, therefore, it is not that easy to call it outside of the ProM framework. Would be cool to have it as a standalone application in the future.

    Until then, you can use the two attached files to call it from the command line. Just copy those files in the ProM (nightly builds) installation directory, make sure the current Log package is installed, and call the CSV importer via the CSV.bat file. It will startup ProM in CLI mode and print out help:


    D:\ProM\Installation\prom-nightly>CSV.bat
    usage: mpe [CSVFILE] [-complete <completionColumn>] [-event <eventColumn>]
           [-h] [-start <startColumn>] [-trace <traceColumn>] [-xes
           <filename>]
     -complete <completionColumn>
     -event <eventColumn>
     -h                             help
     -start <startColumn>
     -trace <traceColumn>
     -xes <filename>



  • DStekel
    DStekel
    Dear Felix,

    I am trying to get this command working, but do not quite know what I should fill in for the parameters. 

    What I have done so far:
    - Installed Prom 6.7 and added the two files you mentioned in the referenced topic to the install directory: (C:\Users\dst\ProM 6.7).

    - Tried to apply the command on one of my test csv files. I assume that the other parameters need to be column names?? Example command:

    C:\Users\dst\ProM 6.7>CSV.bat "C:\Workflow logs\AanvraagFacilitaireDienst.csv" -start "Eind" -event "Taak - Omschrijving" -trace "Dos.item" -xes "C:\Workflow logs\AanvraagFacilitaireDienstNew.xes"

    However, the resulting xes file in not a valid file (Not xml-like when I open it in notepad )

    Furthermore, I retrieve the following error when running: 

    [BEGIN CONSOLE OUTPUT]
    Starting CLI execution engine...
    Parameters: [org.processmining.log.csvimport.CSVConversionCLI, C:\Workflow logs\AanvraagFacilitaireDienst.csv, -start, Eind, -event, Taak - Omschrijving, -trace, Dos.item, -xes, C:\Workflow logs\AanvraagFacilitaireDienstNew.xes]
    Sorting CSV file (0,01 MB) by case and time using maximal 1092 MB of memory ...
    Finished sorting in 0,27 seconds
    Reading cases ...
    java.lang.NoSuchMethodException: org.deckfour.xes.factory.XFactoryNaiveImpl.commit()
            at java.lang.Class.getMethod(Unknown Source)
            at org.processmining.log.csvimport.CSVConversion.commitFactoryIfNeeded(CSVConversion.java:406)
            at org.processmining.log.csvimport.CSVConversion.convertCSV(CSVConversion.java:378)
            at org.processmining.log.csvimport.CSVConversion.doConvertCSVToXES(CSVConversion.java:189)
            at org.processmining.log.csvimport.CSVConversionCLI.parseCSV(CSVConversionCLI.java:139)
            at org.processmining.log.csvimport.CSVConversionCLI.main(CSVConversionCLI.java:88)
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
            at java.lang.reflect.Method.invoke(Unknown Source)
            at org.processmining.cli.CLISimpleStarter.main(CLISimpleStarter.java:34)
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
            at java.lang.reflect.Method.invoke(Unknown Source)
            at org.processmining.framework.boot.Boot.boot(Boot.java:327)
            at org.processmining.framework.boot.Boot.boot(Boot.java:294)
            at org.processmining.cli.CLISimpleStarter.main(CLISimpleStarter.java:41)
    Finished reading cases in 0 seconds.
    start serializing log to XES.XML
    finished serializing log (36 msec.)
    Log converted successfully!
    [END CONSOLE OUTPUT]

    Could you give me some tips on how I should use the command?
  • fmannhardt
    fmannhardt
    Hi, it seems that you have found a bug. Actually that exception should never have been thrown since it is caught in the latest version on the SVN. Would you mind trying with the nightly build version? 

    Not sure if that was broken in ProM 6.7. Sorry about that, but we cannot fix bugs in the released version to ensure results being reproducible (and bugs as well)
  • DStekel
    DStekel
    Thanks for you response!

    After using the nightly-night of 29 January, this is my output:
    [Begin Console Output]
    ...
    Skipping: /C:/Users/dst/ProM-nightly-20180129-1.7/ProM-nightly-20180129-1.7/lib/lpsolve55j.jar while scanning for plugins.
    >>> Scanning for plugins took 21.736 seconds
    Starting CLI execution engine...
    Parameters: [org.processmining.log.csvimport.CSVConversionCLI, C:\Workflow logs\AanvraagFacilitaireDienst.csv, -start, Eind, -event, Taak - Omschrijving, -trace, Dos.item, -xes, C:\Workflow logs\AanvraagFacilitaireDienstNew.xes]
    Sorting CSV file (0,01 MB) by case and time using maximal 1092 MB of memory ...
    Finished sorting in 0,29 seconds
    Reading cases ...
    Finished reading cases in 0 seconds.
    start serializing log to XES.XML
    finished serializing log (25 msec.)
    Log converted successfully!
    [End Console Output]

    Unfortunately, the resulting .xes file contains gibberish: 
    ‹       í][sÚ8~Þþ
    -Oí,˲±ñ$í¤·™Ìô2³¥»}UŒ0JŒÍØ’þúJ&æÒ:Œ!¾(“É#KGú>}çø¢‹7÷ÌIÑÀ¿lÁŽÚÄw‚!õÝËÖ÷ÁGÅj7¯_\ü­(`0¦Q€1ŽÀ
    !>p‰OBÌÈ,(6&àë”ø?>|½ qøÐ×8?
    ÂIåÑâÂ?> )öèOÌx@0Z~Á-DûC¯¼Àå‚»ðÃkKìócâ³ÎúàVõÇîÙ€wo]ä·vn•ú÷ùgAÞy<ÇÔÃ7Ü£0˜€1cS»Û],€º'Q'Ýh°8²í\qdD0›…$ºlù$â¾S0c!½™1µ@bh³oNëõ‹¿.È=#¾8
    |<!—­OtDœÇ#-0
    ɈÞ_¶¼õ¡YH/[MäV•Ä'q;WeÅWÜv«›v’lØgñ™¦E±Vß¾C¦lmØIdÚ~,¹eÞñpÑ%á£ýs>!À,Z{G¢Õ lñý•^®k«/ïÁÊ¿€…˜·^LÐWiöÀÊ•öºäò$Zßu~k˜coÆOz…ýyˆ±û;ԣӐ¼§ÜW¬ãDó¥
    nÓ!üÃÖ4M‡z\‹û\ôF|Ú®wå0J€ÞÒÛȇtÄÖm¹£‰G–ç)÷G\ëÿ ¼yÁbUªüÇ„IÁÍ3¼‚Ђ)£·ÃUy5­ä$œžË0¾ã-ú:‰t;ç߯ê½%ÜÚV›öv)XúÔ‹
    –^M3’ôí©&joéá ÍÔêT|J‡,½OÉ }‹g½`°½†jÓ¯ÊfѪðÆÜú­Ù«ñ݉ÿ³5Κ‰þ¨RHK

    ù’°,%ˆÀx'Óõ4UaOQMµokº
    Q‡›øGÕl5u¢¤,±ûòIO²‡)yrd³¼ø8 7ÙpÑ]á-ò>ÎB4^%±(äó%Ø-„ZÏÈ C™w
    w÷Œ˜ÛéÛÓïš·v6ã^\oÀ¡¼€³ld5p×ëùÀ2ˆ{ŠEO3Ðy—½'!zF¼írm-Ö¹^NØ!Í6`³awJ…©§=k ;Žø(ªWo¡Û=ÊeZÝôS®nq¤5fÅ
    èÕÈ%ËâÃ]3èt¸Þ\r̉ųkúé'˜©¼ÕËÁ4YDuÐ\+ˆ`Žgºõ¤¿è&I³H“Ù™ý¨FöCÓÙ†j«
    Ï~|áíøîNÌõâ±ÛH5†£;2«¦0LóoY¥¡±Ö—†kº
    !‡[}°–{•Ô3VÉM¢ÞV°Hàø,Q.®Ï±¸®Q¶ÑŸO³ˆ—,èíOê ˜ó¸Ìy»`ø†x› 4)HAØ·
    s7ϱ‚°ƒ(¢\D32Y§RõxÒq—¸3ÑÒ‰¼‘àåæh¼Ú‡‹q\£›Üƒõ!´š’ÛºŽÚ†Y„쐩’±Q¡i¯R±ÐA²ê0V‰…òË*$e•”UÏGdz_ʪ¢d•¡×‡ÐjBH‰¬‚yob‘²ªÄl$eUV‰…òÊ*SË™­úNYiLêªjé*ÓÖŒÚ몔KÿE
    ++NÀCqf83Ï= )Ý]
    =æ=˜x+"ÚJJw°ßæo[ºv4ïIÕV:®«’j;ãrË6“ÿJŽkÇY9Ø(‹Ì$3ÇL;Ñ^.¦Ñó3M–S$Ó|"]Í™Ó/)SÉà³ZÁg¯ ÷Jœ8øì+š5€†>µÂn–(†.³‡W
    ³£‚O£m©½¶ÑïËà³ôï`®“ÁgšQa×$ÇU‚ãdôY>jjBô™íI5ÇEŸýÛ5¡*}V+ú´lXÿ[ÊN}B_úìÙºn돩1XºÌ^©ÌŽRf°­jõS'ªŒ>«Íu2úLs¢ZØn$ÇU‚ãdôY>jjBô™íI5GEŸºjÉÇÄeðùŒw|èõ¿ôyÒç™Dì©ÄýhÈÖúEÅžÏOh5!$¨¶-Óh[Fª„’áaµÙ¨JáᙞgÊöa•X(¯¬2“—p×$”ºªRºJíe¾2¦ºê”I}îÌ¥°B¶Ñ·u½TIý=†·"¢­¤tg´{°×Fº|½ô¢ípª«’h;SN_5l­°Í"$ÅUâP꣚2§ÿ¬ÔÔ€œþN‘TsLð‰ Jî„—/HoVyîøîøK/ã‚Ð0¬Ñ~y‘+_Ú,~½b(jüâC³QÆñj5Ú£ÍÛ>m±R’4’¡ª'I8rã'‰Åk×åN~'C^¯ýZdru¼•i+¿
    œgMÚ¨¶¸­ÙV~Ûs¥ì—UŠ­ü´Ô‹Br/¿<ºPCyo#ådQ/Œ3¿†akO¼{S÷Þ>­Òrž“ؐeÉ€·‘UÅ€AqŸR뙃¿èzûúÅ/²H@¦  

    I have no idea why the output file looks like this. Any ideas?
  • DStekel
    DStekel
    Nevermind, I discovered that the resulting xes. file is zipped. When extracting through 7-zip, the file looks correct!

    Thank you fmannhardt!
Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In Register

Categories

In this Discussion