Experiments

8 minute readReference

Experiments have been deprecated and Flags are the only entity in the system.

The system is backwards compatible, but until a new version of the CloudBees Feature Management REST API is released, note the following issues:

  • Create an experiment - creates a corresponding flag, ignoring the experiment name.

  • Get experiment - works as expected, ignoring the experiment name.

  • Patch experiment - works as expected, ignoring the experiment name.

  • Get experiments - returns only flags that have at least one release rule configured.

  • Delete experiment - removes a flag’s configuration and/or impression data, but does NOT delete the flag.

Experiment schema

  • String type - Type of the yaml object; the value has to be experiment

  • String flag - The flag being controlled by this experiment

    • availableValues - The available values that this flag can be

  • String name - The name of the experiment

  • String description - The description of the experiment

  • Boolean enabled - Indicates if the experiment is active

    • labels - Experiment labels

  • String stickinessProperty - Stickiness property that controls percentage based tossing (shows if the default value has been changed)

  • [Array] platforms - Specific platforms configurations

    • String platforms[].name - Name of the platform, as defined in the SDK running

    • String platforms[].flag - Override the flag name, when needed - deprecated

  • [Array] platforms[].conditions - Conditions Array for the specific platform (see Condition schema)

    • platforms[].value - The value when no condition is met for the specific platform

  • [Array] conditions - Default platform conditions (All platforms that have no specific configurations)

    • value - Value when no Condition is met (for the Default platform)

Condition schema

  • value - The value when the Condition is met

  • Object dependency - Condition this flag value with another flag value

  • String dependency.flag - Flag name

  • dependency.value - The value to compare to the flag it is dependent on. For example, if flag2 has a condition of (flag1=== 5) then true, then the dependency.value equals 5.

  • Object group - The condition flag value based on target group(s)

  • String group.operator - The logical relationship between the groups

  • Array group.name - The name of target groups

  • Object version - The condition flag value based release version

  • String version.operator - The operator to compare version

  • String version.semver - The version to compare to

  • Object property - The condition flag value based directly on a custom property

  • String property.name - The name of a custom property

  • String property.operator - The operator to use on the property

  • String/Array/Number/Boolean property.operand - The operand to evaluate the operator with (value to compare with the custom property)

For the full specification, refer to Introduction to Configuration as Code (CasC).

Get Experiments

Returns all experiments

https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/environmentName/experiments

Node
cURL
Ruby
Python
Java
C#
Go
var request = require("request"); var options = { method: 'GET', url: 'https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments', headers: { accept: 'application/json', authorization: 'Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6' } }; request(options, function (error, response, body) { if (error) throw new Error(error); console.log(body); });
curl --request GET \ --url https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments \ --header 'accept: application/json' \ --header 'authorization: Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6'
require 'uri' require 'net/http' require 'openssl' url = URI("https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments") http = Net::HTTP.new(url.host, url.port) http.use_ssl = true http.verify_mode = OpenSSL::SSL::VERIFY_NONE request = Net::HTTP::Get.new(url) request["accept"] = 'application/json' request["authorization"] = 'Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6' response = http.request(request) puts response.read_body
import requests url = "https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments" headers = { 'accept': "application/json", 'authorization': "Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6" } response = requests.request("GET", url, headers=headers) print(response.text)
OkHttpClient client = new OkHttpClient(); Request request = new Request.Builder() .url("https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments") .get() .addHeader("accept", "application/json") .addHeader("authorization", "Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6") .build(); Response response = client.newCall(request).execute();
var client = new RestClient("https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments"); var request = new RestRequest(Method.GET); request.AddHeader("accept", "application/json"); request.AddHeader("authorization", "Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6"); IRestResponse response = client.Execute(request);
package main import ( "fmt" "net/http" "io/ioutil" ) func main() { url := "https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments" req, _ := http.NewRequest("GET", url, nil) req.Header.Add("accept", "application/json") req.Header.Add("authorization", "Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6") res, _ := http.DefaultClient.Do(req) defer res.Body.Close() body, _ := ioutil.ReadAll(res.Body) fmt.Println(res) fmt.Println(string(body)) }

PATH PARAMS

  • applicationId string - The application id

  • environmentName string - The name of the environment

Experiment response

Example:

[ { "type": "experiment", "labels": "Default", "flag": "enableTutorial", "value": [ { "from": "2020-09-25T13:38:29.358Z", "percentage": 0, "option": true }, { "from": "2020-09-25T23:00:00.000Z", "percentage": 100, "option": true } ] }, { "type": "experiment", "labels": "Default", "flag": "titleColors", "availableValues": [ "Blue", "Yellow", "Green", "White" ], "value": [ { "percentage": 25, "option": "Blue" }, { "percentage": 25, "option": "Yellow" }, { "percentage": 25, "option": "Green" }, { "percentage": 25, "option": "White" } ] }, { "type": "experiment", "name": "following views again", "flag": "default.followingView", "conditions": [ { "version": { "operator": "semver-gte", "semver": "3.0.1" }, "group": { "name": [ "QA", "Beta Users" ] }, "value": true } ], "value": false }, { "type": "experiment", "name": "acmeExperimentFoo", "description": "Applies foo to our Nifty App", "labels": "do-not-merge", "stickinessProperty": "fixed", "flag": "acmeFlagControl", "value": false } ]

Response schema type: Experiment schema Array

Create an Experiment

Add a new experiment or override existing one.

https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/environmentName/experiments

Node
cURL
Ruby
Python
Java
C#
Go
var request = require("request"); var options = { method: 'PUT', url: 'https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments', headers: { 'content-type': 'application/json', authorization: 'Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6' }, body: { type: 'experiment', apiVersion: '1.0', flag: 'acmeFlagControl', name: 'acmeExperimentFoo', description: 'Applies foo to our Nifty App', enabled: true, labels: 'do-not-merge', stickinessProperty: 'fixed' }, json: true }; request(options, function (error, response, body) { if (error) throw new Error(error); console.log(body); });
curl --request PUT \ --url https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments \ --header 'authorization: Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6' \ --header 'content-type: application/json' \ --data '{"type":"experiment","apiVersion":"1.0","flag":"acmeFlagControl","name":"acmeExperimentFoo","description":"Applies foo to our Nifty App","enabled":true,"labels":"do-not-merge","stickinessProperty":"fixed"}'
require 'uri' require 'net/http' require 'openssl' url = URI("https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments") http = Net::HTTP.new(url.host, url.port) http.use_ssl = true http.verify_mode = OpenSSL::SSL::VERIFY_NONE request = Net::HTTP::Put.new(url) request["content-type"] = 'application/json' request["authorization"] = 'Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6' request.body = "{\"type\":\"experiment\",\"apiVersion\":\"1.0\",\"flag\":\"acmeFlagControl\",\"name\":\"acmeExperimentFoo\",\"description\":\"Applies foo to our Nifty App\",\"enabled\":true,\"labels\":\"do-not-merge\",\"stickinessProperty\":\"fixed\"}" response = http.request(request) puts response.read_body
import requests url = "https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments" payload = "{\"type\":\"experiment\",\"apiVersion\":\"1.0\",\"flag\":\"acmeFlagControl\",\"name\":\"acmeExperimentFoo\",\"description\":\"Applies foo to our Nifty App\",\"enabled\":true,\"labels\":\"do-not-merge\",\"stickinessProperty\":\"fixed\"}" headers = { 'content-type': "application/json", 'authorization': "Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6" } response = requests.request("PUT", url, data=payload, headers=headers) print(response.text)
OkHttpClient client = new OkHttpClient(); MediaType mediaType = MediaType.parse("application/json"); RequestBody body = RequestBody.create(mediaType, "{\"type\":\"experiment\",\"apiVersion\":\"1.0\",\"flag\":\"acmeFlagControl\",\"name\":\"acmeExperimentFoo\",\"description\":\"Applies foo to our Nifty App\",\"enabled\":true,\"labels\":\"do-not-merge\",\"stickinessProperty\":\"fixed\"}"); Request request = new Request.Builder() .url("https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments") .put(body) .addHeader("content-type", "application/json") .addHeader("authorization", "Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6") .build(); Response response = client.newCall(request).execute();
var client = new RestClient("https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments"); var request = new RestRequest(Method.PUT); request.AddHeader("content-type", "application/json"); request.AddHeader("authorization", "Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6"); request.AddParameter("application/json", "{\"type\":\"experiment\",\"apiVersion\":\"1.0\",\"flag\":\"acmeFlagControl\",\"name\":\"acmeExperimentFoo\",\"description\":\"Applies foo to our Nifty App\",\"enabled\":true,\"labels\":\"do-not-merge\",\"stickinessProperty\":\"fixed\"}", ParameterType.RequestBody); IRestResponse response = client.Execute(request);
package main import ( "fmt" "strings" "net/http" "io/ioutil" ) func main() { url := "https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments" payload := strings.NewReader("{\"type\":\"experiment\",\"apiVersion\":\"1.0\",\"flag\":\"acmeFlagControl\",\"name\":\"acmeExperimentFoo\",\"description\":\"Applies foo to our Nifty App\",\"enabled\":true,\"labels\":\"do-not-merge\",\"stickinessProperty\":\"fixed\"}") req, _ := http.NewRequest("PUT", url, payload) req.Header.Add("content-type", "application/json") req.Header.Add("authorization", "Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6") res, _ := http.DefaultClient.Do(req) defer res.Body.Close() body, _ := ioutil.ReadAll(res.Body) fmt.Println(res) fmt.Println(string(body)) }

PATH PARAMS

  • applicationId string - The application id

  • environmentName* string - The name of the environment that the experiment belongs to

BODY PARAMS

  • Experiment scehma object

Get Experiment

Returns experiment configuration by its flag name.

https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/environmentName/experiments/flagName

Node
cURL
Ruby
Python
Java
C#
Go
var request = require("request"); var options = { method: 'GET', url: 'https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments/acmeFlagControl', headers: { accept: 'application/json', authorization: 'Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6' } }; request(options, function (error, response, body) { if (error) throw new Error(error); console.log(body); });
curl --request GET \ --url https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments/acmeFlagControl \ --header 'accept: application/json' \ --header 'authorization: Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6'
require 'uri' require 'net/http' require 'openssl' url = URI("https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments/acmeFlagControl") http = Net::HTTP.new(url.host, url.port) http.use_ssl = true http.verify_mode = OpenSSL::SSL::VERIFY_NONE request = Net::HTTP::Get.new(url) request["accept"] = 'application/json' request["authorization"] = 'Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6' response = http.request(request) puts response.read_body
import requests url = "https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments/acmeFlagControl" headers = { 'accept': "application/json", 'authorization': "Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6" } response = requests.request("GET", url, headers=headers) print(response.text)
OkHttpClient client = new OkHttpClient(); Request request = new Request.Builder() .url("https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments/acmeFlagControl") .get() .addHeader("accept", "application/json") .addHeader("authorization", "Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6") .build(); Response response = client.newCall(request).execute();
var client = new RestClient("https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments/acmeFlagControl"); var request = new RestRequest(Method.GET); request.AddHeader("accept", "application/json"); request.AddHeader("authorization", "Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6"); IRestResponse response = client.Execute(request);
package main import ( "fmt" "net/http" "io/ioutil" ) func main() { url := "https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments/acmeFlagControl" req, _ := http.NewRequest("GET", url, nil) req.Header.Add("accept", "application/json") req.Header.Add("authorization", "Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6") res, _ := http.DefaultClient.Do(req) defer res.Body.Close() body, _ := ioutil.ReadAll(res.Body) fmt.Println(res) fmt.Println(string(body)) }

PATH PARAMS

  • applicationId string - The application id

  • environmentName string - The name of the environment that the experiment belongs to

  • flagName string - flag of the experiment

Experiment response

Example:

{ "type": "experiment", "name": "acmeExperimentFoo", "description": "Applies foo to our Nifty App", "labels": "do-not-merge", "stickinessProperty": "fixed", "flag": "acmeFlagControl", "value": false }

Response schema type: Experiment

Patch Experiment

Patch Experiment object by flag name, patching is done according to JSON-Patch (RFC6902)

https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/environmentName/experiments/flagName

Node
cURL
Ruby
Python
Java
C#
Go
var request = require("request"); var options = { method: 'PATCH', url: 'https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments/acmeFlagControl', headers: { 'content-type': 'application/json', authorization: 'Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6' }, body: [{op: 'replace', path: '/value', value: true}], json: true }; request(options, function (error, response, body) { if (error) throw new Error(error); console.log(body); });
curl --request PATCH \ --url https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments/acmeFlagControl \ --header 'authorization: Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6' \ --header 'content-type: application/json' \ --data '[{"op":"replace","path":"/value","value":true}]'
require 'uri' require 'net/http' require 'openssl' url = URI("https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments/acmeFlagControl") http = Net::HTTP.new(url.host, url.port) http.use_ssl = true http.verify_mode = OpenSSL::SSL::VERIFY_NONE request = Net::HTTP::Patch.new(url) request["content-type"] = 'application/json' request["authorization"] = 'Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6' request.body = "[{\"op\":\"replace\",\"path\":\"/value\",\"value\":true}]" response = http.request(request) puts response.read_body
import requests url = "https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments/acmeFlagControl" payload = "[{\"op\":\"replace\",\"path\":\"/value\",\"value\":true}]" headers = { 'content-type': "application/json", 'authorization': "Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6" } response = requests.request("PATCH", url, data=payload, headers=headers) print(response.text)
OkHttpClient client = new OkHttpClient(); MediaType mediaType = MediaType.parse("application/json"); RequestBody body = RequestBody.create(mediaType, "[{\"op\":\"replace\",\"path\":\"/value\",\"value\":true}]"); Request request = new Request.Builder() .url("https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments/acmeFlagControl") .patch(body) .addHeader("content-type", "application/json") .addHeader("authorization", "Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6") .build(); Response response = client.newCall(request).execute();
var client = new RestClient("https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments/acmeFlagControl"); var request = new RestRequest(Method.PATCH); request.AddHeader("content-type", "application/json"); request.AddHeader("authorization", "Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6"); request.AddParameter("application/json", "[{\"op\":\"replace\",\"path\":\"/value\",\"value\":true}]", ParameterType.RequestBody); IRestResponse response = client.Execute(request);
package main import ( "fmt" "strings" "net/http" "io/ioutil" ) func main() { url := "https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments/acmeFlagControl" payload := strings.NewReader("[{\"op\":\"replace\",\"path\":\"/value\",\"value\":true}]") req, _ := http.NewRequest("PATCH", url, payload) req.Header.Add("content-type", "application/json") req.Header.Add("authorization", "Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6") res, _ := http.DefaultClient.Do(req) defer res.Body.Close() body, _ := ioutil.ReadAll(res.Body) fmt.Println(res) fmt.Println(string(body)) }

PATH PARAMS

applicationId string The application id

environmentName* string - The name of the environment that the experiment belongs to

flagName* string - flag of the experiment

BODY PARAMS

  • op string - The operation to be performed (one of add, remove, replace, move, copy, test)

  • path string - Path to the value that is to be changed. Ex: path: value is like saying experiment.value

  • value string - The value to be used within the operations.

  • from string - A string containing a JSON Pointer value.

COMMON PATCH BODY EXAMPLES

  • Changing stickiness property

[{"op": "replace", "path": "/stickinessProperty", "value": "groupId"}]

  • Changing Experiment value to true

[{"op": "replace", "path": "/value", "value": true}]

  • Adding a group condition to the end of an experiment’s configurations (assuming the experiment already has conditions)

[{"op": "add", "path": "/conditions/-", "value": {"group":{"operator": "or", "name": "internalEmployees"}, "value": true}}]

  • Adding 2 items to the end of an equal one of array, to the third condition of the experiment

[{"op": "add", "path": "/conditions/2/property/operand/-", "value": "Sokka"}, {"op": "add", "path": "/conditions/2/property/operand/-", "value": "Katara"}]

  • Setting the Experiment to false for the Android platform (assuming there are platforms set, but Android is not)

[{"op": "add", "path": "/platforms/-", "value": [{"name": "Android", "value": false}]}]

Delete Experiment

Removes experiment from the requested environment by its flag name.

https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/environmentName/experiments/flagName

Node
cURL
Ruby
Python
Java
C#
Go
var request = require("request"); var options = { method: 'DELETE', url: 'https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments/acmeFlagControl', headers: {authorization: 'Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6'} }; request(options, function (error, response, body) { if (error) throw new Error(error); console.log(body); });
curl --request DELETE \ --url https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments/acmeFlagControl \ --header 'authorization: Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6'
require 'uri' require 'net/http' require 'openssl' url = URI("https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments/acmeFlagControl") http = Net::HTTP.new(url.host, url.port) http.use_ssl = true http.verify_mode = OpenSSL::SSL::VERIFY_NONE request = Net::HTTP::Delete.new(url) request["authorization"] = 'Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6' response = http.request(request) puts response.read_body
import requests url = "https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments/acmeFlagControl" headers = {'authorization': 'Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6'} response = requests.request("DELETE", url, headers=headers) print(response.text)
OkHttpClient client = new OkHttpClient(); Request request = new Request.Builder() .url("https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments/acmeFlagControl") .delete(null) .addHeader("authorization", "Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6") .build(); Response response = client.newCall(request).execute();
var client = new RestClient("https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments/acmeFlagControl"); var request = new RestRequest(Method.DELETE); request.AddHeader("authorization", "Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6"); IRestResponse response = client.Execute(request);
package main import ( "fmt" "net/http" "io/ioutil" ) func main() { url := "https://x-api.rollout.io/public-api/applications/1a23bcd4e5fg6h7890i123j4/Production/experiments/acmeFlagControl" req, _ := http.NewRequest("DELETE", url, nil) req.Header.Add("authorization", "Bearer b0c17g9f-79f4-69da-cc45-67db602c10d6") res, _ := http.DefaultClient.Do(req) defer res.Body.Close() body, _ := ioutil.ReadAll(res.Body) fmt.Println(res) fmt.Println(string(body)) }

PATH PARAMS

  • applicationId* string - The application ID

  • environmentName* string - The name of the environment that the experiment belongs to

  • flagName* string - The flag of the experiment

As a security measure, a rate limit has been implemented to one request per second, and is based on the requester IP address. Any requests at rates that exceed the rate limit will receive a 555 error status code. There are no exceptions we can currently make for customers at this time.