{"action":"opened","issue":{"url":"https://api.github.com/repos/cloudurable/cassandra-image/issues/43","repository_url":"https://api.github.com/repos/cloudurable/cassandra-image","labels_url":"https://api.github.com/repos/cloudurable/cassandra-image/issues/43/labels{/name}","comments_url":"https://api.github.com/repos/cloudurable/cassandra-image/issues/43/comments","events_url":"https://api.github.com/repos/cloudurable/cassandra-image/issues/43/events","html_url":"https://github.com/cloudurable/cassandra-image/issues/43","id":214852856,"number":43,"title":"more sysctl.conf settings etc","user":{"login":"RichardHightower","id":382678,"avatar_url":"https://avatars1.githubusercontent.com/u/382678?v=3","gravatar_id":"","url":"https://api.github.com/users/RichardHightower","html_url":"https://github.com/RichardHightower","followers_url":"https://api.github.com/users/RichardHightower/followers","following_url":"https://api.github.com/users/RichardHightower/following{/other_user}","gists_url":"https://api.github.com/users/RichardHightower/gists{/gist_id}","starred_url":"https://api.github.com/users/RichardHightower/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/RichardHightower/subscriptions","organizations_url":"https://api.github.com/users/RichardHightower/orgs","repos_url":"https://api.github.com/users/RichardHightower/repos","events_url":"https://api.github.com/users/RichardHightower/events{/privacy}","received_events_url":"https://api.github.com/users/RichardHightower/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2017-03-16T22:08:32Z","updated_at":"2017-03-16T22:08:32Z","closed_at":null,"body":"Linux\r\nsysctl.conf\r\n\r\nFrom ....\r\n\r\nhttps://tobert.github.io/pages/als-cassandra-21-tuning-guide.html\r\n\r\n\r\nCompare to what we have\r\n\r\n```\r\nThe primary interface for tuning the Linux kernel is the /proc virtual filesystem. In recent years, the /sys filesystem has expanded on what /proc does. Shell scripts using echo and procedural code are difficult to manage automatically, as we all know from handling cassandra-env.sh. This led the distros to create /etc/sysctl.conf, and in modern distros, /etc/sysctl.conf.d. The sysctl command reads the files in /etc and applies the settings to the kernel in a consistent, declarative fashion.\r\n\r\nThe following is a block of settings I use almost every server I touch. Most of these are safe to apply live and should require little tweaking from site to site. Note: I have NOT tested these extensively with multi-DC, but most of them should be safe. Those items that may need extra testing for multi-DC have comments indicating it.\r\n\r\nhttp://tobert.github.io/post/2014-06-24-linux-defaults.html\r\n\r\n# The first set of settings is intended to open up the network stack performance by\r\n# raising memory limits and adjusting features for high-bandwidth/low-latency\r\n# networks.\r\nnet.ipv4.tcp_rmem = 4096 87380 16777216\r\nnet.ipv4.tcp_wmem = 4096 65536 16777216\r\nnet.core.rmem_max = 16777216\r\nnet.core.wmem_max = 16777216\r\nnet.core.netdev_max_backlog = 2500\r\nnet.core.somaxconn = 65000\r\nnet.ipv4.tcp_ecn = 0\r\nnet.ipv4.tcp_window_scaling = 1\r\nnet.ipv4.ip_local_port_range = 10000 65535\r\n\r\n# this block is designed for and only known to work in a single physical DC\r\n# TODO: validate on multi-DC and provide alternatives\r\nnet.ipv4.tcp_syncookies = 0\r\nnet.ipv4.tcp_timestamps = 0\r\nnet.ipv4.tcp_sack = 0\r\nnet.ipv4.tcp_fack = 1\r\nnet.ipv4.tcp_dsack = 1\r\nnet.ipv4.tcp_orphan_retries = 1\r\n\r\n# significantly reduce the amount of data the kernel is allowed to store\r\n# in memory between fsyncs\r\n# dirty_background_bytes says how many bytes can be dirty before the kernel\r\n# starts flushing in the background. Set this as low as you can get away with.\r\n# It is basically equivalent to trickle_fsync=true but more efficient since the\r\n# kernel is doing it. Older kernels will need to use vm.dirty_background_ratio\r\n# instead.\r\nvm.dirty_background_bytes = 10485760\r\n\r\n# Same deal as dirty_background but the whole system will pause when this threshold\r\n# is hit, so be generous and set this to a much higher value, e.g. 1GB.\r\n# Older kernels will need to use dirty_ratio instead.\r\nvm.dirty_bytes = 1073741824\r\n\r\n# disable zone reclaim for IO-heavy applications like Cassandra\r\nvm.zone_reclaim_mode = 0\r\n\r\n# there is no good reason to limit these on server systems, so set them\r\n# to 2^31 to avoid any issues\r\n# Very large values in max_map_count may cause instability in some kernels.\r\nfs.file-max = 1073741824\r\nvm.max_map_count = 1073741824\r\n\r\n# only swap if absolutely necessary\r\n# some kernels have trouble with 0 as a value, so stick with 1\r\nvm.swappiness = 1\r\nOn vm.max_map_count:\r\n\r\nhttp://linux-kernel.2935.n7.nabble.com/Programs-die-when-max-map-count-is-too-large-td317670.html\r\n\r\nlimits.conf (pam_limits)\r\nThe DSE and DSC packages install an /etc/security/limits.d/ file by default that should remove most of the problems around pam_limits(8). Single-user systems such as database servers have little use for these limitations, so I often turn them off globally using the following in /etc/security/limits.conf. Some users may already be customizing this file, in which case change all of the asterisks to cassandra or whatever the user DSE/Cassandra is running as.\r\n\r\n* - nofile     1000000\r\n* - memlock    unlimited\r\n* - fsize      unlimited\r\n* - data       unlimited\r\n* - rss        unlimited\r\n* - stack      unlimited\r\n* - cpu        unlimited\r\n* - nproc      unlimited\r\n* - as         unlimited\r\n* - locks      unlimited\r\n* - sigpending unlimited\r\n* - msgqueue   unlimited\r\n```"}}
{"actor":{"display_login":"RichardHightower"}}