Hi @marcp, the FMEFeature.getAttribute(<list attribute name including {}>) method returns a list containing string elements, and every <null> element within the original list attribute will be interpreted as the empty string. I don't think the method returns the None even if the first element was <null>.
However, if the first element was <missing> (i.e. the first element does not exist), the method determines that the list itself does not exist, and then returns the None. I guess that this is the situation you have observed. Be aware that <null> and <missing> are differentiated strictly in FME 2014+.
If you can treat the <missing> element as a specific value (e.g. the empty string), consider using the NullAttributeMapper to assign the value to missing 'From_Address' for every feature before building the list attribute by the LineJoiner.
Or if you need to preserve <missing> and <null> as-is, you will have to get the maximum index of the list attribute and then retrieve each element value with the getAttributeNullMissingAndType method rather than getAttribute. See this simplified example.
import re
def processFeature(feature):
maxIndex = -1
for attr in feature.getAllAttributeNames():
m = re.match('^_list\{(\d+)\}\.from$', attr)
if m:
i = int(m.group(1))
if maxIndex < i:
maxIndex = i
if maxIndex < 0:
print ('_list{}.from does not exist')
else:
for i in range(maxIndex + 1):
attr = '_list{%d}.from' % i
isNull, isMissing, t = feature.getAttributeNullMissingAndType(attr)
if isNull:
print ('_list{%d}.from stores <null>' % i)
elif isMissing:
print ('_list{%d}.from is <missing>' % i)
else:
value = feature.getAttribute(attr)
print ('_list{%d}.from stores \'%s\'' % (i, value))
Note: this script can be applied to FME 2014 SP1 or later.
When you call feature.getAttribute() on a list attribute, it will assume that your list starts with element 0 and that it increases sequentially without gaps. That means this list is ok:
addr{0} = 'a'
addr{1} = 'b'
addr{2} = 'c'
But this list isn't considered a valid list by FME (does not start with 0), and feature.getAttribute() will return a single instance of None, like if it didn't even exist at all:
addr{1} = 'a'
addr{2} = 'b'
addr{3} = 'c'
This list will result in feature.getAttribute() only returning the first two elements, since there is a gap in the numbering:
addr{0} = 'a'
addr{1} = 'b'
addr{3} = 'c'
If it can happen that your list attribute starts with a non-zero index and/or you have gaps in your numbers, you will have to fix that first or do some regex-magic (see Takashi's answer here) to extract the list manually.
When you have extracted your FME list attribute into a Python list, you can then do a simple check to see if you have a list with some elements in it or not, like this:
if addr:
print "FME list 'addr{}' exists and has at least one element"
else:
print "FME list 'addr{}' does not exist or is empty"
When you call feature.getAttribute() on a list attribute, it will assume that your list starts with element 0 and that it increases sequentially without gaps. That means this list is ok:
addr{0} = 'a'
addr{1} = 'b'
addr{2} = 'c'
But this list isn't considered a valid list by FME (does not start with 0), and feature.getAttribute() will return a single instance of None, like if it didn't even exist at all:
addr{1} = 'a'
addr{2} = 'b'
addr{3} = 'c'
This list will result in feature.getAttribute() only returning the first two elements, since there is a gap in the numbering:
addr{0} = 'a'
addr{1} = 'b'
addr{3} = 'c'
If it can happen that your list attribute starts with a non-zero index and/or you have gaps in your numbers, you will have to fix that first or do some regex-magic (see Takashi's answer here) to extract the list manually.
When you have extracted your FME list attribute into a Python list, you can then do a simple check to see if you have a list with some elements in it or not, like this:
if addr:
print "FME list 'addr{}' exists and has at least one element"
else:
print "FME list 'addr{}' does not exist or is empty"
Agree. In more detail, seems that the getAttribute method collects the list elements from consecutive index sequence that starts with 0 if a list name is specified. With this experimental script:
def processFeature(feature):
L = feature.getAttribute('_list{}')
if L == None:
print ('the list does not exist or the first element is missing')
else:
print (L)
- When the list contains _list{0} = 0, _list{1} = 1, and _list{3} = 3 (_list{2} is missing), L will contain r'0', '1'] but '3' will be dropped.
- When the list contains _list{1} = 1, _list{2} = 2, and _list{3} = 3 (_list{0} is missing), L will be None.
- Naturally the method returns None if the specified list does not exist (all elements are missing).
If the From_Address attribute may be missing in the source features, all the case 1, 2, and 3 could occur. I therefore think it's better to resolve the 'missing' situation with the NullAttributeMapper beforehand.
Agree. In more detail, seems that the getAttribute method collects the list elements from consecutive index sequence that starts with 0 if a list name is specified. With this experimental script:
def processFeature(feature):
L = feature.getAttribute('_list{}')
if L == None:
print ('the list does not exist or the first element is missing')
else:
print (L)
- When the list contains _list{0} = 0, _list{1} = 1, and _list{3} = 3 (_list{2} is missing), L will contain i'0', '1'] but '3' will be dropped.
- When the list contains _list{1} = 1, _list{2} = 2, and _list{3} = 3 (_list{0} is missing), L will be None.
- Naturally the method returns None if the specified list does not exist (all elements are missing).
If the From_Address attribute may be missing in the source features, all the case 1, 2, and 3 could occur. I therefore think it's better to resolve the 'missing' situation with the NullAttributeMapper beforehand.
One more thing, if you have a list like this:
addr{0}.From_Address = 'a'
addr{1}.From_Address = 'b'
addr{1}.To_Address = 'c'
And you do the following in Python:
feature.getAttribute('addr{}.To_Address')
This will return a single instance of None, rather than the list cNone, 'c'], which is what you might expect (hope).
I think this is unfortunate. This could have been avoided by implementing this idea: https://knowledge.safe.com/content/idea/21655/fmeobjects-support-for-complex-lists.html
So everybody who reads this, please vote for it :-)
Hi @marcp, the FMEFeature.getAttribute(<list attribute name including {}>) method returns a list containing string elements, and every <null> element within the original list attribute will be interpreted as the empty string. I don't think the method returns the None even if the first element was <null>.
However, if the first element was <missing> (i.e. the first element does not exist), the method determines that the list itself does not exist, and then returns the None. I guess that this is the situation you have observed. Be aware that <null> and <missing> are differentiated strictly in FME 2014+.
If you can treat the <missing> element as a specific value (e.g. the empty string), consider using the NullAttributeMapper to assign the value to missing 'From_Address' for every feature before building the list attribute by the LineJoiner.
Or if you need to preserve <missing> and <null> as-is, you will have to get the maximum index of the list attribute and then retrieve each element value with the getAttributeNullMissingAndType method rather than getAttribute. See this simplified example.
import re
def processFeature(feature):
maxIndex = -1
for attr in feature.getAllAttributeNames():
m = re.match('^_list\{(\d+)\}\.from$', attr)
if m:
i = int(m.group(1))
if maxIndex < i:
maxIndex = i
if maxIndex < 0:
print ('_list{}.from does not exist')
else:
for i in range(maxIndex + 1):
attr = '_list{%d}.from' % i
isNull, isMissing, t = feature.getAttributeNullMissingAndType(attr)
if isNull:
print ('_list{%d}.from stores <null>' % i)
elif isMissing:
print ('_list{%d}.from is <missing>' % i)
else:
value = feature.getAttribute(attr)
print ('_list{%d}.from stores \'%s\'' % (i, value))
Note: this script can be applied to FME 2014 SP1 or later.
Hi @takashi, you are right and my situation is a list that starts with a MISSING value.
Based on your example I've creating the following which works perfectly in my scenario. This didn't require the NullAttributeMapper, but relies on the fact that there are attributes which will always have a value, in this case the index.
def processFeature(feature):
nn = feature.getAttribute('Full_Name')
if nn == 'Bedford Rd':
pieces = len( feature.getAttribute('C:\GIS\SK\Saskatoon\Roads\Roads.sl3{}.Road_ID'))
print nn, pieces,
for i in range(pieces):
print '%d:' %i,
try:
attr = 'C:\GIS\SK\Saskatoon\Roads\Roads.sl3{%d}.From_Address'%i
isNull, isMissing, t = feature.getAttributeNullMissingAndType(attr)
if not (isNull or isMissing):
print feature.getAttribute(attr),
else:
print '<missing>',
except:
print '{', i, '}',
print ""<br>
I have to say that I didn't understand the first part of your code, are you just confirming that the list attribute actually exists?
thanks,
When you call feature.getAttribute() on a list attribute, it will assume that your list starts with element 0 and that it increases sequentially without gaps. That means this list is ok:
addr{0} = 'a'
addr{1} = 'b'
addr{2} = 'c'
But this list isn't considered a valid list by FME (does not start with 0), and feature.getAttribute() will return a single instance of None, like if it didn't even exist at all:
addr{1} = 'a'
addr{2} = 'b'
addr{3} = 'c'
This list will result in feature.getAttribute() only returning the first two elements, since there is a gap in the numbering:
addr{0} = 'a'
addr{1} = 'b'
addr{3} = 'c'
If it can happen that your list attribute starts with a non-zero index and/or you have gaps in your numbers, you will have to fix that first or do some regex-magic (see Takashi's answer here) to extract the list manually.
When you have extracted your FME list attribute into a Python list, you can then do a simple check to see if you have a list with some elements in it or not, like this:
if addr:
print "FME list 'addr{}' exists and has at least one element"
else:
print "FME list 'addr{}' does not exist or is empty"
Hi @david_r,
It seems that getAttribute is a 'little bit' broken in the case of lists.
Your answer added some context that I didn't quite get beforehand.. Lists with <missing> have non-sequential indexes. Now I get it, thanks.
I think your last point above is actually false. A list with no element in index 0 returns None, as you state earlier, so the 'if addr' test returns false, even though the actual list contains elements.
Hi @takashi, you are right and my situation is a list that starts with a MISSING value.
Based on your example I've creating the following which works perfectly in my scenario. This didn't require the NullAttributeMapper, but relies on the fact that there are attributes which will always have a value, in this case the index.
def processFeature(feature):
nn = feature.getAttribute('Full_Name')
if nn == 'Bedford Rd':
pieces = len( feature.getAttribute('C:\GIS\SK\Saskatoon\Roads\Roads.sl3{}.Road_ID'))
print nn, pieces,
for i in range(pieces):
print '%d:' %i,
try:
attr = 'C:\GIS\SK\Saskatoon\Roads\Roads.sl3{%d}.From_Address'%i
isNull, isMissing, t = feature.getAttributeNullMissingAndType(attr)
if not (isNull or isMissing):
print feature.getAttribute(attr),
else:
print '<missing>',
except:
print '{', i, '}',
print ""<br>
I have to say that I didn't understand the first part of your code, are you just confirming that the list attribute actually exists?
thanks,
The first part in my script gets the maximum index of the list based on the elements which exist. The intention is equivalent to the use of 'pieces' in your script.
Hi @david_r,
It seems that getAttribute is a 'little bit' broken in the case of lists.
Your answer added some context that I didn't quite get beforehand.. Lists with <missing> have non-sequential indexes. Now I get it, thanks.
I think your last point above is actually false. A list with no element in index 0 returns None, as you state earlier, so the 'if addr' test returns false, even though the actual list contains elements.
I'm not sure I agree that the last point is false, but I do agree I could've made it clearer by saying that the list was empty according to FME/fmeobjects. Also, I guess one could ask the philosophical question whether a list that isn't recognized by FME is still a list at all, or just some attributes with fancy names :-)