Question

FME Server Error when Submitting a Job

  • 7 March 2019
  • 6 replies
  • 50 views
R

Environment - Ubuntu 16.04 LTS Docker Host, running FME Server 2018.1.1-20190122 images

 

I am receiving an error - There was an error submitting the job - when trying to run a workspace within FME Sever. Digging through the FME Server logs I find the following error within the Resources > Logs > tomcat > fmeserverweb > localhost current log.

07-Mar-2019 09:05:10.300 SEVERE [http-nio-8080-exec-1] org.apache.catalina.core.StandardWrapperValve.invoke Servlet.service() for servlet [JobRunnerForm] in context with path [/fmeserver] threw exception
 java.net.ConnectException: Connection refused (Connection refused)
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:673)
    at COM.safe.web.client.SelfSignedSSLSocketFactory.connectSocket(SelfSignedSSLSocketFactory.java:52)
    at org.apache.http.conn.scheme.SchemeSocketFactoryAdaptor.connectSocket(SchemeSocketFactoryAdaptor.java:66)
    at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
    at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:144)
    at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:131)
    at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
    at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
    at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:117)
    at COM.safe.web.client.HTTPClientFacade.execute(HTTPClientFacade.java:254)
    at COM.safe.fmeserver.webui.jobrun.JobRunnerHelper.runJob(JobRunnerHelper.java:240)
    at COM.safe.fmeserver.webui.servlet.rest.JobRunnerFormREST.handleRequest(JobRunnerFormREST.java:182)
    at COM.safe.fmeserver.webui.servlet.rest.JobRunnerFormREST.doPost(JobRunnerFormREST.java:67)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:661)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at COM.safe.web.servlet.CharsetEncodingFilter.doFilter(CharsetEncodingFilter.java:541)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at COM.safe.webservices.security.FMEServerAuthFilter.doFilter(FMEServerAuthFilter.java:98)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.ebaysf.web.cors.CORSFilter.handleNonCORS(CORSFilter.java:640)
    at org.ebaysf.web.cors.CORSFilter.doFilter(CORSFilter.java:173)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:493)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
    at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:650)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:800)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:800)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1471)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:748)

I have stood up the exact same FME Server environment on anther machine and I do not receive this error and I am able to run a workspace without issue. The only difference between the environments are the SSL certificates being used by the Nginx container. I have tried using a self signed SSL certificate and not one I have generated, but this does not seem to help the issue either.

Do you have any idea what might be causing this issue. It seems to be an SSL error with an internal system? I am not sure what else to check at this point. The SSL certificates I have been using are still valid (not expired). 

 

Thanks

Randy

6 replies

rylanatsafe
Rank
Badge +11

Two recently reported issues exhibited the same symptom and there were two resolutions that worked:

1. Remove or update the old SSL Certs .crt and .key, and restart the containers.

2. Specify a different hostname for FME Server using EXTERNALHOSTNAME when starting building or starting the containers – suggested to try using an IP Address.

Note: I recognize this is an old posting, but I wanted to add this information in case it helps any other users. Please add a comment if either of the suggestions above helped!

dejan11
Rank
Badge +7

Two recently reported issues exhibited the same symptom and there were two resolutions that worked:

1. Remove or update the old SSL Certs .crt and .key, and restart the containers.

2. Specify a different hostname for FME Server using EXTERNALHOSTNAME when starting building or starting the containers – suggested to try using an IP Address.

Note: I recognize this is an old posting, but I wanted to add this information in case it helps any other users. Please add a comment if either of the suggestions above helped!

Hi @rylanatsafe​ ,

I am facing the very similar problem on windows.

In my case, I have successfully certificated server and it works, but when I run workspace, triggers following:

21-Aug-2022 15:14:32.328 SEVERE [https-openssl-apr-443-exec-20] org.apache.catalina.core.StandardWrapperValve.invoke Servlet.service() for servlet [JobRunnerForm] in context with path [/fmeserver] threw exception
	java.net.ConnectException: Connection refused: connect
		at java.net.DualStackPlainSocketImpl.connect0(Native Method)
		at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:79)
		at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
		at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
		at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
		at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
		at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
		at java.net.Socket.connect(Socket.java:607)
		at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:681)
		at COM.safe.web.client.SelfSignedSSLSocketFactory.connectSocket(SelfSignedSSLSocketFactory.java:52)
		at org.apache.http.conn.scheme.SchemeSocketFactoryAdaptor.connectSocket(SchemeSocketFactoryAdaptor.java:66)
		at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
		at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:144)
		at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:134)
		at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:605)
		at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:440)
		at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:835)
		at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118)
		at COM.safe.web.client.HTTPClientFacade.execute(HTTPClientFacade.java:254)
		at COM.safe.fmeserver.webui.jobrun.JobRunnerHelper.runJob(JobRunnerHelper.java:193)
		at COM.safe.fmeserver.webui.servlet.rest.JobRunnerFormREST.handleRequest(JobRunnerFormREST.java:149)
		at COM.safe.fmeserver.webui.servlet.rest.JobRunnerFormREST.doPost(JobRunnerFormREST.java:61)
		at javax.servlet.http.HttpServlet.service(HttpServlet.java:681)
		at javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
		at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227)
		at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
		at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
		at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
		at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
		at COM.safe.web.servlet.CharsetEncodingFilter.doFilter(CharsetEncodingFilter.java:541)
		at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
		at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
		at COM.safe.webservices.security.FMEServerAuthFilter.doFilter(FMEServerAuthFilter.java:130)
		at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
		at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
		at org.ebaysf.web.cors.CORSFilter.handleSimpleCORS(CORSFilter.java:467)
		at org.ebaysf.web.cors.CORSFilter.doFilter(CORSFilter.java:161)
		at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
		at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
		at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:197)
		at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
		at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:660)
		at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135)
		at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
		at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:687)
		at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
		at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:360)
		at org.apache.coyote.http2.StreamProcessor.service(StreamProcessor.java:426)
		at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
		at org.apache.coyote.http2.StreamProcessor.process(StreamProcessor.java:87)
		at org.apache.coyote.http2.StreamRunnable.run(StreamRunnable.java:35)
		at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
		at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
		at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
		at java.lang.Thread.run(Thread.java:748)

I suppose EXTERNALHOSTNAME relates to docker.

What else would you suggest to check?

I have followed following steps: https://community.safe.com/s/article/Using-a-CA-Certificate-with-FME-Server, except first 6 steps ( and last 14-18 steps), because I have downloaded private.key, intermediate and certificate from CA and updated Connector in server.xml as follows:

    <Connector port="443" protocol="org.apache.coyote.http11.Http11AprProtocol"
               maxThreads="150" SSLEnabled="true" >
        <UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" />
        <SSLHostConfig>
            <Certificate certificateKeyFile="C:\Program Files\FMEServer\Utilities\tomcat\conf\private.key"
                         certificateFile="C:\Program Files\FMEServer\Utilities\tomcat\conf\certificate.crt"
                         certificateChainFile="C:\Program Files\FMEServer\Utilities\tomcat\conf\ca_bundle.crt"
                         type="RSA" />
        </SSLHostConfig>
    </Connector>

I have also checked the following:

https://community.safe.com/s/article/run-a-service-from-web-ui-sample-job-fails

and imported as trusted:

https://community.safe.com/s/article/pkix-path-issues-when-configuring-ssl-for-fme-serv

but no help.

Any help would be very appreciated. 

Thank you,

Dejan

 

Dejan
merlinegeorge
Rank
Badge +5

Hello @dejan11​ ,

 

Our documentation for "Configuring FME Server for HTTPS: Using a CA Certificate" mentions a different <Connector>  element for step "8. Configure Server.xml".

I see that you are setting this up using a different connector element - imageWould you be able to use the .pem format for all the files instead of the .key and .crt?

dejan11
Rank
Badge +7

Hi @Merline George​ ,

Yes, I was able to convert .crt, *.key file to .pem and configure, but it is the same error.

I think there is something in FME Server configuration missing and I cannot find what exacly.

The certificate is valid both tried as crt, key or pem format.

 

Dejan
merlinegeorge
Rank
Badge +5

Hello @dejan11​ ,

It might be best to submit a case at this point so as to look deeper into the FME Server config files or additional log files.

dejan11
Rank
Badge +7

Hello @dejan11​ ,

It might be best to submit a case at this point so as to look deeper into the FME Server config files or additional log files.

Just did, thanks.

Dejan

Reply


Community Stats

28,661
Posts
109,324
Replies
36,528
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.