Skip to main content

Hi everyone,

 

 

i have a simple workbench that uses the ShortestPathFinder to create paths and i have around 1,000,000 from-to lines to query.

Before I run the From-To lines through the workbench I made sure that all line vertices are snapped to the nearest network node, and then within this workbench I snapped them again by checking the 'From-To and Network Snapping' option in the ShortestPathFinder transformer.

Somehow, the workbench worked I when i send through a sample of From-To lines (around 100 lines), but failed completely when i send through the whole 1,000,000 From-To lines.

 

 

The error is - the source/destination node in the shortest path calculation is not a valide node in the network.

Did it failed because I am sending through too mahy queries at once? Is there a way to get around this?

Thanks

Rings

 

 

Hi @ring_a I've never tried 1M From-To lines but it should work in theory. If the error message is accurate it sounds like a problem with a particular From-To line. Try sending a different group of 100 or 1000 From-To lines to see if you can narrow down the problem. If you get this error message every time then please report your problem here.


Hi @ring_a,

 

I might have run into this before when my input lines actually formed multiple valid networks. Have you run your From-To lines through a NetworkTopologyCalculator? I wonder if you get more than one feature out of this transformer.

 

Hope this helps,

Nathan


Hi @ring_a,

 

I might have run into this before when my input lines actually formed multiple valid networks. Have you run your From-To lines through a NetworkTopologyCalculator? I wonder if you get more than one feature out of this transformer.

 

Hope this helps,

Nathan

Thanks Nathan,

I do have multiple networks within my network. is there a way to extract the single largest network?

Rings


Hi @ring_a I've never tried 1M From-To lines but it should work in theory. If the error message is accurate it sounds like a problem with a particular From-To line. Try sending a different group of 100 or 1000 From-To lines to see if you can narrow down the problem. If you get this error message every time then please report your problem here.

Thanks Dan,

After splitting my lines into groups, I seem to get this problem occasionally. I can't see anything wrong with the From-To lines as there are all valid lines generated from vertex creaters and using the same coordinate system as the network. I feel like the problem comes from the network.

Rings


How complicated are your from_to lines? I've seen the error before but i can't recall the circumstance. Normally if it's an issue with the from-to line you'd just get data output via the NoRoute port with a more specific error message "The source node at (0, 5) in the shortest path calculation is not a valid node in the network" so i agree it sounds like something else


Thanks Nathan,

I do have multiple networks within my network. is there a way to extract the single largest network?

Rings

Hi @ring_a,

I suppose one way would be to calculate the length of all the lines coming out of the NetworkTopologyCalculator, then aggregate by the network_id and sum. Sort and sample the largest number. Or if your lines are already all the same length, just count the number of lines in each network_id group and take the largest number.

 

 

Another option might be to try to determine why you have multiple networks, and see if it is feasible to connect them. But that might be more of a manual or at least "supervised" iterative process.

 

Hope that helps,

 

Nathan

Hi @ring_a,

I suppose one way would be to calculate the length of all the lines coming out of the NetworkTopologyCalculator, then aggregate by the network_id and sum. Sort and sample the largest number. Or if your lines are already all the same length, just count the number of lines in each network_id group and take the largest number.

 

 

Another option might be to try to determine why you have multiple networks, and see if it is feasible to connect them. But that might be more of a manual or at least "supervised" iterative process.

 

Hope that helps,

 

Nathan

hi Nathan,

I managed to extract the largest network by fanning out through network id. But unfortunately, having rerun the shortestpathfinder with this new network, i still get the same errors.

Any other thoughts?

cheers,

rings


Hi @ring_a I've never tried 1M From-To lines but it should work in theory. If the error message is accurate it sounds like a problem with a particular From-To line. Try sending a different group of 100 or 1000 From-To lines to see if you can narrow down the problem. If you get this error message every time then please report your problem here.

Dan,

you might be right about that the problem being a few from-to lines. I tried a few other groups but haven't yet narrowed down the problem. I will try reporting the problem to safe.

 

many thanks,

 

rings

hi Nathan,

I managed to extract the largest network by fanning out through network id. But unfortunately, having rerun the shortestpathfinder with this new network, i still get the same errors.

Any other thoughts?

cheers,

rings

Are you sure that every start/end node of the From-To lines exactly matches a node belonging to the largest network you have extracted?


Hello @ring_a

 

What is your 'Snapping Tolerance' set to? Are you certain your source/destination nodes are within this tolerance?

Check that your network vertices are split at intersections, if not run a TopologyBuilder before the ShortestPathFinder (not 100% sure this matters).

Cheers

 

Brian

Can you put a Chopper transformer into the Network stream of data and set Maximum Vertices to 2? A trap I just fell for is that your from-to lines must meet the END node of a line. They won't snap to a vertex in the middle of a line. The Chopper would resolve that.


Reply