Question

How to run fme.exe?

  • 28 October 2021
  • 7 replies
  • 72 views

Userlevel 1
Badge +8

Whenever running a workspace the log start with pointing out how to run the workspace at the command-line. No doubt it works when I open the command-line and run the workspace.

My issue though is that I have a little ui in C# where I do not call cmd.exe, but go with Process.Start() and start directly fme.exe.

My approach works as well, however, it takes 4 minutes to run the workspace in cmd.exe and 30 minutes when calling directly fme.exe.

 

Is there an explanation why it takes so much more time when calling fme.exe instead running it through cmd.exe?


7 replies

Userlevel 4

The first step would be to compare the log files between the two execution types and see what is different.

Userlevel 1
Badge +8

Actually, there is not much of a difference. On both occasions I run the same workspace. Running it via cmd.exe the working directory is located at the location of the workspace, while starting it in fme.exe through my C# application the working directory is within the applications directory. Just at one point something that takes only seconds when running it via cmd.exe,

2021-10-28 11:08:17|  3.2| 0.0|INFORM|Matcher (MatchingFactory): Splitting bulk features into individual features
 
2021-10-28 11:08:18|  3.2| 0.0|INFORM|object_kommune_intersect_m_timezone_SQLExecutor_3 (QueryFactory): 22 input initiators processed
 
2021-10-28 11:08:18|  3.2| 0.0|INFORM|object_kommune_intersect_m_timezone_SQLExecutor (QueryFactory): 22 input initiators processed
 
2021-10-28 11:08:18|  3.2| 0.0|INFORM|PGVInit_med_kommunevalg_SQLExecutor (QueryFactory): 22 input initiators processed

takes about 10 minutes when running directly in fme.exe

2021-10-28 12:16:12|  3.1| 0.0|INFORM|Matcher (MatchingFactory): Splitting bulk features into individual features
 
2021-10-28 12:26:07|  4.8| 1.7|INFORM|PGVInit_med_kommunevalg_SQLExecutor (QueryFactory): Processed 35 of 47 features

Thereafter, it goes relatively smoothly further and takes about the same time, independent on how I execute the workspace.

 

Userlevel 4
Badge +26

Actually, there is not much of a difference. On both occasions I run the same workspace. Running it via cmd.exe the working directory is located at the location of the workspace, while starting it in fme.exe through my C# application the working directory is within the applications directory. Just at one point something that takes only seconds when running it via cmd.exe,

2021-10-28 11:08:17|  3.2| 0.0|INFORM|Matcher (MatchingFactory): Splitting bulk features into individual features
 
2021-10-28 11:08:18|  3.2| 0.0|INFORM|object_kommune_intersect_m_timezone_SQLExecutor_3 (QueryFactory): 22 input initiators processed
 
2021-10-28 11:08:18|  3.2| 0.0|INFORM|object_kommune_intersect_m_timezone_SQLExecutor (QueryFactory): 22 input initiators processed
 
2021-10-28 11:08:18|  3.2| 0.0|INFORM|PGVInit_med_kommunevalg_SQLExecutor (QueryFactory): 22 input initiators processed

takes about 10 minutes when running directly in fme.exe

2021-10-28 12:16:12|  3.1| 0.0|INFORM|Matcher (MatchingFactory): Splitting bulk features into individual features
 
2021-10-28 12:26:07|  4.8| 1.7|INFORM|PGVInit_med_kommunevalg_SQLExecutor (QueryFactory): Processed 35 of 47 features

Thereafter, it goes relatively smoothly further and takes about the same time, independent on how I execute the workspace.

 

And in both cases you're Running on the same machine? It looks like the performance hit is when accessing the database. In the fast case the request here seems instant. But in the second case a 10 min jump looks more like FME is waiting on the database to respond?

 

Are you also using the same version of FME to run it?

Userlevel 2
Badge +19

I have coded some apps to achieve the same as you and I haven't noticed any impact in performance.

 

This is my code in case it helps:

string strCmdText  = '"' + [path_to_workspace] + '"';
 Process process = new Process();
 process.StartInfo.FileName = [path_to_fme.exe]
 process.StartInfo.Arguments = strCmdText;
 process.StartInfo.UseShellExecute = false;
 process.StartInfo.RedirectStandardOutput = true;
 process.StartInfo.RedirectStandardError = true;
 process.Start();
 
 string output = process.StandardOutput.ReadToEnd();
 string err = process.StandardError.ReadToEnd();
 process.WaitForExit();

 

Userlevel 1
Badge +8

And in both cases you're Running on the same machine? It looks like the performance hit is when accessing the database. In the fast case the request here seems instant. But in the second case a 10 min jump looks more like FME is waiting on the database to respond?

 

Are you also using the same version of FME to run it?

Yes, it is at the same computer, using the same version of FME. While debugging I am listening to the ProcessStartInfo object to see what is called, and what parameters handed over. So, yes it's the same computer, the same network connection, the same FME version - JUST running fme.exe in cmd vs. fme.exe directly.

Userlevel 1
Badge +8

I have coded some apps to achieve the same as you and I haven't noticed any impact in performance.

 

This is my code in case it helps:

string strCmdText  = '"' + [path_to_workspace] + '"';
 Process process = new Process();
 process.StartInfo.FileName = [path_to_fme.exe]
 process.StartInfo.Arguments = strCmdText;
 process.StartInfo.UseShellExecute = false;
 process.StartInfo.RedirectStandardOutput = true;
 process.StartInfo.RedirectStandardError = true;
 process.Start();
 
 string output = process.StandardOutput.ReadToEnd();
 string err = process.StandardError.ReadToEnd();
 process.WaitForExit();

 

Thanks, doesn't look promising, but reassuring... I am doing exactly what you are doing @oscard​ 

ProcessStartInfo startInfo = new ProcessStartInfo()
            {
                FileName = fmeExe,
                UseShellExecute = false,
                RedirectStandardError = true,
                RedirectStandardOutput = true
            };
 
string arguments =
                    $"{scriptName} " +
                    $"--kommune \"{SelectedMunicipality.Code}\" " +
                    $"--geodb \"C:\\Temp{output}.gdb.zip\" " +
                    $"--user \"{user}\"";
 
 using (Process process = Process.Start(startInfo))
                {
                    if (process == null) return;
                    try
                    {
                        //get FmeScript from collection to update values
                        FmeScript fmw = FmeScripts.FirstOrDefault(x => x.FmwPath == startInfo.Arguments.Split(new[] { ' ' }, 2)[0]);
                        fmw.ProcessStart = process.StartTime;
                        process.ErrorDataReceived += process_ErrorDataReceived;
                        process.BeginErrorReadLine();
                        process.WaitForExit(600000);
                        if (process.HasExited)
                        {
                            fmw.ProcessEnd = process.ExitTime;
                            fmw.Status = _status == null ? "succesfuld" : _status;
                            await AddScriptToTable(fmw, user, taskId, fmeVersion);
                            _status = null;
                        }
                    }
                    catch (Exception e)
                    {
                        Console.WriteLine(e);
                        throw;
                    }
                }

 

Userlevel 4
Badge +26

And in both cases you're Running on the same machine? It looks like the performance hit is when accessing the database. In the fast case the request here seems instant. But in the second case a 10 min jump looks more like FME is waiting on the database to respond?

 

Are you also using the same version of FME to run it?

Yeah that's super strange - Perhaps a different user? I've seen that permissions can effect certain db operations, although that has usually been specific to ESRI formats. Another tool which could help is the ProcessMonitor (procmon.exe) to monitor the fme.exe job to see if you can identify any other differences.

If different users, it could be that FME is using some cache or a local version of a file?

Reply