Solved

Get available disk space programmatically using FME Server API?

  • 1 August 2022
  • 4 replies
  • 14 views

Badge

Does anyone know if there is a way to get the amount of free disk space using the FME Server API? I presume I could use a SystemCaller transformer and write out the information somewhere and access it that way, but wondering if there is an easier way. Just looking to get the same disk space details that are logged when a job runs (see screenshot). Thanks

icon

Best answer by virtualcitymatt 9 August 2022, 14:22

View original

4 replies

Badge +6

Hi @jorma​ ,

 

There isn't an endpoint available for pulling information about the host machine resources, that I'm aware of. The SystemCaller might be the best bet here, or something like low disk alerts on Windows: https://docs.microsoft.com/en-us/troubleshoot/windows-server/backup-and-storage/configure-low-disk-space-alert-performance-logs

 

We can set up FME to trigger clean-up events, based on disk space: https://docs.safe.com/fme/html/FME_Server_Documentation/WebUI/Configuring-System-Cleanup-Settings.htm

 

To find the disk space as recorded on each job, I'd recommend using the Retrieve parsed job log request, GET /transformations/jobs/id/< jobid >/log, then parsing those highlighted lines.

 

For FME Cloud, it's all much simpler: https://docs.safe.com/fme_cloud/FME_Cloud/Content/About_Instances/Instance-Monitoring.htm

 

Lastly, not entirely what you're looking for, but we can monitor job statistics in FME Server 2021.0+: https://community.safe.com/s/article/Monitoring-FME-Server-Job-Activity-using-the-REST-API

 

Hope that helps!

 

Sanae

Userlevel 4
Badge +26

You can use python to get that info easy enough:

imagemodified from https://stackoverflow.com/questions/48929553/get-hard-disk-size-in-python

 

import fme
import fmeobjects
import shutil
 
def GetSize(object):
    
    total, used, free = shutil.disk_usage("/")
 
    object.setAttribute("Total",(total // (2**30)))
    object.setAttribute("Used",(used // (2**30)))
    object.setAttribute("Free",(free // (2**30)))

 

Badge

Hi @jorma​ ,

 

There isn't an endpoint available for pulling information about the host machine resources, that I'm aware of. The SystemCaller might be the best bet here, or something like low disk alerts on Windows: https://docs.microsoft.com/en-us/troubleshoot/windows-server/backup-and-storage/configure-low-disk-space-alert-performance-logs

 

We can set up FME to trigger clean-up events, based on disk space: https://docs.safe.com/fme/html/FME_Server_Documentation/WebUI/Configuring-System-Cleanup-Settings.htm

 

To find the disk space as recorded on each job, I'd recommend using the Retrieve parsed job log request, GET /transformations/jobs/id/< jobid >/log, then parsing those highlighted lines.

 

For FME Cloud, it's all much simpler: https://docs.safe.com/fme_cloud/FME_Cloud/Content/About_Instances/Instance-Monitoring.htm

 

Lastly, not entirely what you're looking for, but we can monitor job statistics in FME Server 2021.0+: https://community.safe.com/s/article/Monitoring-FME-Server-Job-Activity-using-the-REST-API

 

Hope that helps!

 

Sanae

Thanks @sanaeatsafe​ , appreciate all the info. The solution provided below works pretty good, so I think I am all set.

Badge

You can use python to get that info easy enough:

imagemodified from https://stackoverflow.com/questions/48929553/get-hard-disk-size-in-python

 

import fme
import fmeobjects
import shutil
 
def GetSize(object):
    
    total, used, free = shutil.disk_usage("/")
 
    object.setAttribute("Total",(total // (2**30)))
    object.setAttribute("Used",(used // (2**30)))
    object.setAttribute("Free",(free // (2**30)))

 

Works perfect. Thanks @virtualcitymatt​ !

Reply