Info

The memory size is taken into account within various components today: We set the default WiredTiger cache_size to approximately 1/2 of the memory size When ephemeral storage is in use we consider the memory limit in place to determine the default oplog size We set the tcmalloc cache size to 1/8th of the available memory In all of these cases we should take the actual memory constraint that we're operating under into account rather than the total system memory – for example when running MongoDB within containers.

Top User Comments

xgen-internal-githook commented on Wed, 6 Mar 2019 02:55:07 +0000: Author: {'name': 'Matt Lord', 'username': 'mattlord', 'email': 'mattalord@gmail.com'} Message: Revert "SERVER-16571 cache_size should be set based on cgroup available RAM not physical RAM" This reverts commit a39875e4e060d42a7ce70ec82b07af2850d3bab7. Branch: master https://github.com/mongodb/mongo/commit/dbef14de7332bb910200518e80a2b130d2b973f4 xgen-internal-githook commented on Wed, 6 Mar 2019 02:55:04 +0000: Author: {'name': 'Matt Lord', 'username': 'mattlord', 'email': 'mattalord@gmail.com'} Message: Revert "SERVER-16571 Apply linter fixes to original fix" This reverts commit 602bfb9c52b2274d55492f73eeac8513d9048d10. Branch: master https://github.com/mongodb/mongo/commit/d654fe781da630ce7f354ec31931efc58757e690 xgen-internal-githook commented on Tue, 5 Mar 2019 15:05:34 +0000: Author: {'name': 'Eric Milkie', 'email': 'milkie@10gen.com', 'username': 'milkie'} Message: SERVER-16571 Apply linter fixes to original fix Branch: master https://github.com/mongodb/mongo/commit/602bfb9c52b2274d55492f73eeac8513d9048d10 xgen-internal-githook commented on Tue, 5 Mar 2019 14:18:20 +0000: Author: {'name': 'Matt Lord', 'email': 'mattalord@gmail.com', 'username': 'mattlord'} Message: SERVER-16571 cache_size should be set based on cgroup available RAM not physical RAM Branch: master https://github.com/mongodb/mongo/commit/a39875e4e060d42a7ce70ec82b07af2850d3bab7 james.broadhead commented on Fri, 1 Mar 2019 10:23:57 +0000: acm / matt.lord just noticed that this one ended up on the backlog – is there any chance of getting it for MDB 4.2? (we have a workaround in the Operator, but it's not ideal) matt.lord commented on Tue, 10 Jul 2018 15:21:04 +0000: Hi All, WiredTiger uses the output of ProcessInfo.getMemSizeMB() when determining the cache size, which in turn calls SystemInfo.memSize. On linux, we use LinuxSysHelper::getSystemMemorySize() which is implemented using /proc/meminfo. If we focus on Linux for now (Windows is the only other OS with native kernel [docker] containers today), then the correct way would seem to be: Check to see if our execution context / shell has a memory cgroup applied: NO: grep memory /proc/self/cgroup 8:memory:/ YES: grep memory /proc/self/cgroup 5:memory:/docker/3fd47f54d37af11b6e4144706bb38509adacb10bd08fa419eb4b1e599c016022 If NO: then we use the byte value derived from /proc/meminfo as we always do today. If YES: In getSystemMemorySize() we use the lower of the two values when comparing /sys/fs/cgroup/memory/memory.limit_in_bytes with the byte size value derived from /proc/meminfo Does anyone disagree or have any other comments? This particular implementation would be relatively easy to implement–my concerns here revolve more around testing this on all combinations of: Supported Linux platforms (distro+version+arch) Docker version Am I missing anything? Thanks! Matt anton.lisovenko commented on Mon, 2 Jul 2018 09:36:56 +0000: This ticket is now more important when Mongodb Kubernetes Operator is released to beta and more customers will try Mongodb in containers. Is it possible to revisit the ticket and schedule it if it's possible to get fixed now? ramon.fernandez commented on Wed, 29 Jul 2015 20:05:48 +0000: daldoyle, having a way to work with cgroups is definitely desirable. I'd recommend you post on the mongodb-dev group with an outline of the approach you'd like to take, so other developers can comment on it, ask questions, etc. I'm told that the libcgroup license may be incompatible with MongoDB's, so this issue should be sorted out as well. Regards, Ramón. daldoyle commented on Thu, 23 Jul 2015 20:09:50 +0000: If a patch were provided that made this work only with cgroups in linux, would that be accepted and put into mainstream? The lack of resource isolation is starting to make running mongo in production problematic for us when things like OOM start to kill them off if other things on the box start grabbing memory, so we are trying to figure out some options. I realize this might not be perfect and wouldn't solve the problems with ulimits, but with the growing popularity of cgroups and things like Docker that use them, we're hoping that this might be attractive enough to warrant including. dan@10gen.com commented on Mon, 12 Jan 2015 00:06:20 +0000: It's not currently possible to detect the container memory in a foolproof way it seems, but will leave open for when and if it does become possible. See: http://fabiokung.com/2014/03/13/memory-inside-linux-containers/

Steps to Reproduce