Skip to main content

I have created a fme workbench. It has creator transformer, python caller and S3 Uploader. The purpose of the workbench is to created a pdf (python script does it) and to upload that pdf to S3. 

Our FME is 2019.2.  I am using custom python3 installation. 

If I just run creator and python caller, all works fine. If I disable python caller, and run creator and uploader, all works fine as well. If all parts are connected, I am getting an error:

Python Exception <ImportError>: cannot import name 'ssl' from 'urllib3.util.ssl_' (D:\Program Files\FME2019.2.3\python\urllib3\util\ssl_.py)

PythonFactory failed to load python symbol `fmepy_amazons3.connector.S3Connector'

Factory proxy not initialized

S3Connector_CALLER (PythonFactory): PythonFactory failed to process feature

A fatal error has occurred. Check the logfile above for details

Bridge failed to output feature on tag `PYOUTPUT'

PythonCaller (PythonFactory): PythonFactory failed to process feature

 

In all cases, python script creates output pdf and I can see that it runs to the end (I have print statements). 

 

# Template Function interface:
# When using this function, make sure its name is set as the value of
# the 'Class or Function to Process Features' transformer parameter

def processFeature(feature):

    file_location = FME_MacroValuesÂ'pdf']

    folder = r'https://publiccontent-gis-psba-qld-gov-au.s3.amazonaws.com/content/PDFs/'
    file_name = file_location.rsplit('\\', 1)w1]
    file_path = file_location.rsplit('\\', 1) 0]
    file_link = folder + file_name
    aws_link = file_link.replace(" ", "%20") # for qr coding
    aws_folder = FME_MacroValues1'S3_Path'] #'content/PDFs/' # for S3 upload
    aws_bucket = FME_MacroValuesl'S3Bucket'] #'publiccontent-gis-psba-qld-gov-au'
    temp_path = 'D:\\Temp\\'
    output_pdf = os.path.join(temp_path, file_name) # output pdf for final document
    
    print ("file path done")
    # create qr code image
    qr = pyqrcode.create(aws_link)
    qr.svg(temp_path +'qr_svg.svg', scale=2, background="white", module_color="#000000")

    # convert svg to pdf
    drawing = svg2rlg(temp_path+'qr_svg.svg')
    renderPDF.drawToFile(drawing, temp_path+'qr_pdf.pdf')
    print ("this part done")

    
    input1 =  PdfFileReader(open(file_location, 'rb'), strict=False)
    map_size = input1.getPage(0).mediaBox
    input2 = PdfFileReader(open(temp_path +'qr_pdf.pdf', 'rb'), strict=False)
    qrcode_size = input2.getPage(0).mediaBox

    # merge pdf map and qr code pdf
    # function for merging two pdfs
    def create_qrCode(input_pdf, qr_code, output_pdf):
        qr_obj = PdfFileReader(qr_code, strict=False)
        qr_page = qr_obj.getPage(0)
        pdf_reader = PdfFileReader(input_pdf, strict=False)
        pdf_writer = PdfFileWriter()
        # QrCode all the pages
        for page in range(pdf_reader.getNumPages()):
            page = pdf_reader.getPage(page)
            page.mergePage(qr_page)
            pdf_writer.addPage(page)
        with open(output_pdf, 'wb') as out:
            pdf_writer.write(out)
    create_qrCode(file_location, temp_path +'qr_pdf.pdf', output_pdf)
    print("qr code file create")

    feature.setAttribute("file_out", output_pdf)
    print("script is done - ")

0684Q00000ArDPNQA3.png

0684Q00000ArDPSQA3.png

Could you run the workspace again (separated for both Transformers) and review python interpreter being loaded in each case?

I'm guessing a different interpreter is being loaded depending on which Transformer is encountered first. Share the outcome please.

 


Could you run the workspace again (separated for both Transformers) and review python interpreter being loaded in each case?

I'm guessing a different interpreter is being loaded depending on which Transformer is encountered first. Share the outcome please.

 

I missed something 'using a custom python3 install'.

Looks like it is missing some of the mods that is needed by the s3Connector and can't load the ones from FME... Is there a reason you need to use the custom python3 and not the python3 that comes with FME?

What is your workspace scripting parameter set to for Python Compatibility?


I missed something 'using a custom python3 install'.  

Looks like it is missing some of the mods that is needed by the s3Connector and can't load the ones from FME... Is there a reason you need to use the custom python3 and not the python3 that comes with FME?

What is your workspace scripting parameter set to for Python Compatibility?

I am using custom python3 because I am importing custom libraries. I did not include this in my original post.

import fme
import fmeobjects
from PyPDF2 import PdfFileWriter, PdfFileReader
import PIL
from PIL import Image 
import pyqrcode, io, os, sys
from reportlab.graphics import renderPDF, renderPM
from svglib.svglib import svg2rlg

This my FME preferred Python Interpreter:

 

0684Q00000ArNDzQAN.png

 

0684Q00000ArN8QQAV.png


Could you run the workspace again (separated for both Transformers) and review python interpreter being loaded in each case?

I'm guessing a different interpreter is being loaded depending on which Transformer is encountered first. Share the outcome please.

 

How do I see which python interpreter is loaded?


How do I see which python interpreter is loaded?

@lidiad

Can you review the FME log file, it should indicated it in there... but I think it is likely loading the custom one.

Hence it is not able to load the FME python necessary for the S3Connector.

 

I'm wondering if the log file might show the attempts. Can you share the log file here? If not it might be good to open a case for this if you haven't already.

Reply