BuildStream infrastructure

GitLab CI runners

The BuildStream project uses its own runners sponsored by Codethink to run the CI for performance reasons. The current infrastructure allocated to BuildStream for CI is defined in the following table:

IP / Hostname	Hosted At	CPU	RAM	Disk	Runner Name	Notes
167.99.219.68	Digital Ocean	1vCPU	1GB	25GB	digitalocean-bastion-x86_64 and buildstream-bastion-overnight	Bastion server for Gitlab Autoscale
	Digital Ocean	2vCPU	4GB	25GB		Created on demand by the digitalocean-bastion-x86_64
	Digital Ocean	4vCPU	8GB	160GB		Created on demand by the buildstream-bastion-overnight
	Codethink	8vCPU	64GB	16GB	sled17 to sled24	8x aarch64 sleds
bst-artifacts-gnome.codethink.co.uk	Codethink	4vCPU	16GB	2TB	codethink-x86_64-1	Disabled at the moment
ostree-gnome.codethink.co.uk~	Codethink	4vCPU	8GB	120GB	codethink-x86_64-2	Disabled at the moment

The Digital Ocean machines are managed by a BuildStream group account. A Digital Ocean Storage Space is being used to store the GitLab CI cache.

There are also eight Moonshot Sleds which are currently unused.

Access to all these machines can be provided upon request by ValentinDavid and JavierJardon.

Current GitLab Runner configuration

# cat /etc/gitlab-runner/config.toml
concurrent = 20
check_interval = 0

[[runners]]
  name = "buildstream-bastion"
  output_limit = 32768
  url = "https://gitlab.com/"
  token = "$TOKEN"
  executor = "docker+machine"
  [runners.docker]
    tls_verify = false
    image = "debian:stretch"
    privileged = true
    disable_cache = false
    volumes = ["/cache"]
    shm_size = 0
  [runners.cache]
    Type = "s3"
    ServerAddress = "ams3.digitaloceanspaces.com"
    AccessKey = "$ACCESS_KEY"
    SecretKey = "$SECRET_KEY"
    BucketName = "buildstream-ci-gitlab-cache"
  [runners.machine]
    IdleCount = 0
    IdleTime = 1800
    MachineDriver = "digitalocean"
    MachineName = "gitlab-runner-autoscale-%s"
    MachineOptions = ["digitalocean-image=fedora-28-x64", "digitalocean-ssh-user=root", "digitalocean-access-token=$ACCESS_TOKEN", "digitalocean-region=ams3", "digitalocean-size=c-2", "digitalocean-private-networking"]
    OffPeakTimezone = ""
    OffPeakIdleCount = 0
    OffPeakIdleTime = 0

[[runners]]
  name = "buildstream-bastion-overnight"
  output_limit = 262144
  url = "https://gitlab.com/"
  token = "$TOKEN"
  executor = "docker+machine"
  [runners.docker]
    tls_verify = false
    image = "debian:stretch"
    privileged = true
    disable_cache = true
    volumes = ["/cache"]
    shm_size = 0
  [runners.cache]
    Type = "s3"
    ServerAddress = "ams3.digitaloceanspaces.com"
    AccessKey = "$ACCESS_KEY"
    SecretKey = "$SECRET_KEY"
    BucketName = "buildstream-ci-gitlab-cache"
  [runners.machine]
    IdleCount = 0
    IdleTime = 0
    MachineDriver = "digitalocean"
    MachineName = "gitlab-runner-autoscale-%s"
    MachineOptions = ["digitalocean-image=fedora-28-x64", "digitalocean-ssh-user=root", "digitalocean-access-token=$ACCESS_TOKEN", "digitalocean-region=ams3", "digitalocean-size=s-4vcpu-8gb", "digitalocean-private-networking"]
    OffPeakTimezone = ""
    OffPeakIdleCount = 0
    OffPeakIdleTime = 0

Projects/BuildStream/Infrastructure (last edited 2018-12-19 17:23:51 by JavierJardon)