Split percentage

How split by percentage works

CloudBees Rollout utilizes a distributed algorithm. This algorithm is similar to tossing a coin on the device. But once a coin is tossed the result is consistent for an experiment.

How the algorithm works

  • The client side SDK generates a UUID (called rox.distinct_id) and serializes it on the device.

  • Every experiment has its own seed to make sure there isn’t a statistical bias between 2 experiments.

  • The SDK takes the seed and distinct_id and runs a hash function that is mapped to a double number from 0-1.

If you select a 50% split, for example, then we are checking if the hash result is smaller the 0.5.

Implication of the algorithm

  • The result is consistent, every time a user opens the app they will get the same result (unless you changed percentage).

  • If you increase percentage, all your existing users that already received true value will keep getting it.

  • If you decrease percentage, some of your users will stop receiving a true value even if they have received one before.

  • A single user’s place in the bucket is different in every experiment (so your users will not always get a new feature).

Override distinct_id with user id

You can choose to override the distinct_id. This is important for 2 reasons:

  • Having a consistent cross platform behavior for split

  • Having a consistent experience for your users of backend SDK

See also Flag context.