[Solved] how to find all distance combinations between locations in pyspark [duplicate]

amro_ghoneim Asks: how to find all distance combinations between locations in pyspark [duplicate]
I have data in the following format

Code:
agent_id, client_id, client_long, client_lat
1, 1, ,39.777982,-7.004599
1, 2, ,39.677982,-7.094599
1, 3, ,39.577982,-7.084599
2, 4, ,39.477982,-7.074599
2, 5, ,39.377982,-7.064599

I want to get the average distance between the clients for each agent

so I need to get the distances between clients 1,2,3 (all combinations) for agent 1 and distances between clients 4 and 5 for agent 2 then average these distances for each agent.

How do I go about doing this using pyspark?

Ten-tools.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your response here to help other visitors like you. Thank you, Ten-tools.