BuildStream infrastructure

GitLab CI runners

The BuildStream project uses its own runners sponsored by Codethink to run the CI for performance reasons. The current infrastructure allocated to BuildStream for CI is defined in the following table:

IP / Hostname

Hosted At

CPU

RAM

Disk

Runner Name

Notes

167.99.219.68

Digital Ocean

1vCPU

1GB

25GB

digitalocean-bastion-x86_64 and buildstream-bastion-overnight

Bastion server for Gitlab Autoscale

Digital Ocean

2vCPU

4GB

25GB

Created on demand by the digitalocean-bastion-x86_64

Digital Ocean

4vCPU

8GB

160GB

Created on demand by the buildstream-bastion-overnight

bst-artifacts-gnome.codethink.co.uk

Codethink

4vCPU

16GB

2TB

codethink-x86_64-1

Disabled at the moment

ostree-gnome.codethink.co.uk

Codethink

4vCPU

8GB

120GB

codethink-x86_64-2

Disabled at the moment

The Digital Ocean machines are managed by a BuildStream group account. A Digital Ocean Storage Space is being used to store the GitLab CI cache.

There are also eight Moonshot Sleds which are currently unused.

Access to all these machines can be provided upon request by TiagoGomes and JavierJardon.

Current GitLab Runner configuration

# cat /etc/gitlab-runner/config.toml
concurrent = 20
check_interval = 0

[[runners]]
  name = "buildstream-bastion"
  output_limit = 32768
  url = "https://gitlab.com/"
  token = "$TOKEN"
  executor = "docker+machine"
  [runners.docker]
    tls_verify = false
    image = "debian:stretch"
    privileged = true
    disable_cache = false
    volumes = ["/cache"]
    shm_size = 0
  [runners.cache]
    Type = "s3"
    ServerAddress = "ams3.digitaloceanspaces.com"
    AccessKey = "$ACCESS_KEY"
    SecretKey = "$SECRET_KEY"
    BucketName = "buildstream-ci-gitlab-cache"
  [runners.machine]
    IdleCount = 0
    IdleTime = 1800
    MachineDriver = "digitalocean"
    MachineName = "gitlab-runner-autoscale-%s"
    MachineOptions = ["digitalocean-image=fedora-28-x64", "digitalocean-ssh-user=root", "digitalocean-access-token=$ACCESS_TOKEN", "digitalocean-region=ams3", "digitalocean-size=c-2", "digitalocean-private-networking"]
    OffPeakTimezone = ""
    OffPeakIdleCount = 0
    OffPeakIdleTime = 0

[[runners]]
  name = "buildstream-bastion-overnight"
  output_limit = 262144
  url = "https://gitlab.com/"
  token = "$TOKEN"
  executor = "docker+machine"
  [runners.docker]
    tls_verify = false
    image = "debian:stretch"
    privileged = true
    disable_cache = true
    volumes = ["/cache"]
    shm_size = 0
  [runners.cache]
    Type = "s3"
    ServerAddress = "ams3.digitaloceanspaces.com"
    AccessKey = "$ACCESS_KEY"
    SecretKey = "$SECRET_KEY"
    BucketName = "buildstream-ci-gitlab-cache"
  [runners.machine]
    IdleCount = 0
    IdleTime = 0
    MachineDriver = "digitalocean"
    MachineName = "gitlab-runner-autoscale-%s"
    MachineOptions = ["digitalocean-image=fedora-28-x64", "digitalocean-ssh-user=root", "digitalocean-access-token=$ACCESS_TOKEN", "digitalocean-region=ams3", "digitalocean-size=s-4vcpu-8gb", "digitalocean-private-networking"]
    OffPeakTimezone = ""
    OffPeakIdleCount = 0
    OffPeakIdleTime = 0

Projects/BuildStream/Infrastructure (last edited 2018-11-15 15:52:46 by JavierJardon)