Question

Well, i’m designing a small affiliate system using PHP and Neo4J in order to teach myself a little more about Neo4J and how graph databases work. My original affiliate system was in MySQL and the performance gain by just migrating the design and storage to Neo4J is absurd.

However, there are some features that i’m probably doing VERY wrongly and i’m willing to see how better it could be, with some help from more experienced developers. For example, each month the system has to calculate the amount of dollars to pay to each affiliate. This follows some rules that force me to search through an entire user network.

For example, in order to calculate Joe’s monthly pay, the system has to:

Find Node Joe
function Calculate:
  Calculate amount of people referred (all tiers)
  Calculate the number of referral tiers (all tiers)
  Calculate points
  Store information inside object properties

• tiers: Joe refers Mary who in it’s turn refers Bob. Mary is Tier 1, Bob is Tier 2, under Joe.
• all tiers: calculate number based on Joe, Mary and Bob

So, the Calculate function looks like a recursive function. Foreach user referred by Joe, i have to run the Calculate function on him, and any of his children, until i get back to Joe and get my numbers.

This is slow as hell. I thought about using RabbitMQ or ZMQ and creating a queue for each “children process” of Joe’s calculation. I also thought about using pcntl’s forking. How can i make this recursive process inside my graph network better? What’s the best way to go through the entire tree? Queuing? Process Forking?

Another example:

Calculate(Joe)
  Joe referred Mary, Bob, Peter
  Calculate(Mary)
  Mary referred Sara, Megan
    Calculate(Sara)
    Calculate(Megan)
  Calculate(Bob)
  Bob referred Billy, Michael
    Calculate(Billy)
    Calculate(Michael)
  Calculate(Peter)
  Peter referred Charles, Max
    Calculate(Charles)
    Calculate(Max)

Now, multiplicate this for someone with 500 referrals, and each of his referrals can have more 500 people. It’s slow as hell and i have to show this on Joe’s Dashboard, each month. 🙁

Answer 1

with Neo4j and Cypher, you could do something along the lines of this Graph Gist: http://tinyurl.com/7vryzwz

basically, the Cypher ( http://docs.neo4j.org/chunked/snapshot/cypher-query-lang.html) query is

START referrer=node(1)
MATCH path=referrer-[:referred*1..]->refferee
RETURN referrer,refferee, length(path) as generation
ORDER BY length(path) asc

Does that help?

/peter