GoCD hardware specifications

    The agent hardware requirements are basically determined by the builds that are run. Running GoCD agents will introduce a slight CPU overhead (usually it can be neglected when comparing to the build process CPU requirements). The agent will require an additional memory of about 500Mb. Although, you may run the agent on the same machine as the server, it is recommended that you use a separate machine (though, it may be virtual) for each build agent. If you chose to install several agents on the same machine, please consider possible CPU, disk, memory or network bottlenecks that might occur.

    The GoCD Server is responsible for performing a number of tasks and we describe how each of these tasks affects the specifications:

    Number of web-requests to the server

    Web-requests to the server come from different kinds of sources

    • browsers requests from end users (most pages in GoCD continuously poll the server every few seconds to check if the page has updates on it)
    • API calls from programs
    • CCTray which is popularly used by dashboard apps and other desktop apps to notify devs about failing builds usually poll every few seconds and are fairly expensive calls on the server

    GoCD uses a shared thread-pool to manage a lot of web requests from several users, if you have a lot of users, consider adding more cores based on the number and type of request you serve per minute.

    You can find out the number of requests that your server is handling by turning on web-request logging.

    Number of agents connected to a GoCD server, and how often they build

    Each GoCD agent that is connected to the server will poll the GoCD server every few seconds and to update its status on the server and check if there are any jobs that it should build.

    When a build starts off on an agent, it will first perform an SCM checkout operation to update to the correct revision that must be built. If the job requires that artifacts from other dependent jobs be downloaded, the agent will download the artifacts from the server. During a job run, the agent will send the console logs every few seconds and at the end of the build, an agent may upload artifacts to the server if it is configured to do so.

    To cope up with the number of agents, artifact uploads and downloads, ensure that the server has -

    • enough disk space to store the artifacts
    • enough consistent IO baseline throughput to be able to handle simultaneous uploads and downloads from several agents at the same time
    • enough cores to be able to handle web requests from agents

    Number of source control materials

    GoCD uses up-to 10 processes (configurable) to check if updates are available for source control materials(SCM). Only materials that have polling turned on will be polled. Each material is checked approximately every minute for to see if there are updates available so that GoCD can trigger a build.

    If you have hundreds of SCMs, depending on the performance of your source control updates, the update checks for SCMs may queue up and negatively affect the performance.

    Ensure that -

    • your SCM server(s) can handle the load of several SCM checkout and update operations (both from the server and the agent)
    • there is enough IOPS, IO throughput, memory and cores available to ensure that you can poll all your SCMs without the update checks queueing up, or eating away too many resources that would otherwise be used by the GoCD server process.

    You can find out the amount of time it takes to perform an SCM update check by turning on performance logging on your GoCD server.

    If you find that material updates are taking too much of your CPU time, you may try one or more of these options -

    • turn off polling for materials that don’t update frequently
    • reduce the interval between polls
    • investigate if upgrading the SCM software version resolves the issue

    Type of source control materials, some types perform better than others

    All SCMs need to connect to a remote SCM repository to check if updates are available. Some of them are more efficient at update checks than others.

    SCMs like git and mercurial require a local clone to be able to check if there are new commits and the contents of those commits, as such they will have additional overhead in terms of IO and disk space usage.

    On the other hand, SCMs like SVN and TFS don’t require a local checkout to check for updates, so they may not have a disk IO overhead on the gocd server. However the number and frequency of since a material update check may cause a bottleneck on the network.

    Number of times each of the pipeline has run (pipeline history)

    Each pipeline run (along with the stages and jobs in it) is recorded in the database for purpose of audits. To ensure optimal performance of your GoCD server we recommend that you run your database server on a separate instance

    Number and types of plugins that may be installed

    Each plugin that is installed on the server will consume additional memory on the memory and may have an impact on the CPU utilization of the server. Task plugins, will consume additional on the memory usage of the gocd agent process and may affect the CPU usage of the gocd agent process.

    For the purpose of the rest of this document, we would like to use the following definitions:

    Cores

    The number of physical compute units available to the server machine.

    Memory

    JVM Memory

    The amount of memory available to the JVM running the GoCD server process (roughly the JVM argument)

    RAM

    The amount of memory available to the server machine. Generally speaking, the amount of RAM should be atleast 2-3GB more than the JVM memory to allow some headroom for the operating system.

    Buffer Cache

    Reading from a disk is very slow as compared to reading from memory. Additionally, it is common to read the same part of a disk several times during relatively short periods of time. For example, consider how often the command git might be run on a GoCD server. By reading the information from disk only once and then keeping it in memory until no longer needed, one can speed up all but the first read. This is called disk buffering, and the memory used for the purpose is called the buffer cache.

    Generally speaking, you should ensure that there is sufficient free memory for buffer cache to ensure optimal performance and reduced disk IO.

    In the example above, the first line says that the amount of free memory is “278” mb, this makes it look like the system is running short of memory. However the second line() shows that in reality 2427 mb is actually free. This is because about 20+2128 megs of memory is buffers + cache, which the OS can purge if it is programs request for more memory.

    IOPS

    The amount of input/output operations per seconds of your storage device.

    IO throughput