Linux Operating System Configuration

    We recommend to not use BTRFS on linux, as user have reported issues using it inconjunction with ArangoDB. We experienced ArangoDB facing latency issues when accessing its database files on BTRFS partitions. In conjunction with BTRFS and AUFS we also saw data loss on restart.

    Page Sizes

    On Linux ArangoDB uses asits memory allocator. Unfortunately, some OS configurations can interfere with jemalloc’s ability to function properly. Specifically, Linux’s “transparent hugepages”, Windows’ “large pages” and other similar features sometimes prevent jemalloc from returning unused memory to the operating system and result in unnecessarily high memory use. Therefore, we recommend disabling these features when using jemalloc with ArangoDB. Please consult your operating system’s documentation for how to do this.

    Execute:

    before executing .

    Swap Space

    It is recommended to assign swap space for a server that is running arangod.Configuring swap space can prevent the operating system’s OOM killer fromkilling ArangoDB too eagerly on Linux.

    The recommended kernel setting for overcommit_memory for both MMFiles andRocksDB storage engine is 0 or 1. The Linux kernel default is 0.

    You can set it as follows before executing arangod:

    1. sudo bash -c "echo 0 >/proc/sys/vm/overcommit_memory"

    From www.kernel.org:

    • When this flag is 0, the kernel attempts to estimate the amountof free memory left when userspace requests more memory.

    • When this flag is 2, the kernel uses a “never overcommit”policy that attempts to prevent any overcommit of memory.

    Experience has shown that setting overcommit_memory to a value of 2 has severe negative side-effects when running arangod, so it should be avoided at all costs.

    Max Memory Mappings

    Linux kernels by default restrict the maximum number of memory mappings of asingle process to about 64K mappings. While this value is sufficient for mostworkloads, it may be too low for a process that has lots of parallel threadsthat all require their own memory mappings. In this case all the threads’ memory mappings will be accounted to the single arangod process, and the maximum number of 64K mappings may be reached. When the maximum number ofmappings is reached, calls to mmap will fail, so the process will think nomore memory is available although there may be plenty of RAM left.

    To avoid this scenario, it is recommended to raise the default value for themaximum number of memory mappings to a sufficiently high value. As a rule ofthumb, one could use 8 times the number of available cores times 8,000.

    For a 32 core server, a good rule-of-thumb value thus would be 2,048,000 (32 8 8000). For certain workloads, it may be sensible to use even a highervalue for the number of memory mappings.

    To set the value once, use the following command before starting arangod:

    To make the settings durable, it will be necessary to store the adjusted settings in /etc/sysctl.conf or other places that the operating system islooking at.

    Memory Limits

    A long-running arangod process may accumulate a lot of virtual memory after a while, so it is recommended to not restrict the amount of virtual memory that a process can acquire, neither via using , cgroups or systemd.

    Execute

    1. sudo bash -c "echo 0 >/proc/sys/vm/zone_reclaim_mode"

    From :

    This is value ORed together of

    • 1 = Zone reclaim on
    • 2 = Zone reclaim writes dirty pages out
    • 4 = Zone reclaim swaps pages

    NUMA

    Multi-processor systems often have non-uniform Access Memory (NUMA). ArangoDBshould be started with interleave on such system. This can be achieved using

    Open Files Limit

    An arangod instance may need to use a lot of file descriptors for working withfiles and network resources. It is therefore required to allow arangod processesto use a lot of file descriptors at the same time. A reasonable value for themaximum number of open file descriptors for an arangod process is 1048576. Thisshould provide enough headroom so that arangod doesn’t run out of file descriptors.

    The maximum number of file descriptors can be adjusted using ulimit, and systemd.

    It is recommended to set the environment variable GLIBCXX_FORCE_NEW to 1 onsystems that use glibc++ in order to disable the memory pooling built intoglibc++. That memory pooling is unnecessary because Jemalloc will already domemory pooling.

    Execute

    1. export GLIBCXX_FORCE_NEW=1

    before starting arangod.

    32bit

    While it should be possible to compile ArangoDB on 32bit system, this is not arecommended environment. 64bit systems can address a significantly largermemory region. This is also the reason why only 64bit release builds are suppliedby ArangoDB Inc.