Info

When I try to run below command in the shell, _./mongodump -h localhost:27017 -d dfnbi_config -o "/home/directfn/AML/mongodb/bin/dump/d" --gzip_ shell is showing {{Failed: error connecting to db server: no reachable servers}} later, when I check the mongo log, I can find an exception in the log . 2019-07-04T12:41:05.734+0300 I CONTROL [main] ***** SERVER RESTARTED ***** 2019-07-04T12:41:05.743+0300 I CONTROL [initandlisten] MongoDB starting : pid=6670 port=27017 dbpath=/home/dfnuat/data/mongodb 64-bit host=localhost 2019-07-04T12:41:05.743+0300 I CONTROL [initandlisten] db version v3.4.2 2019-07-04T12:41:05.743+0300 I CONTROL [initandlisten] git version: 3f76e40c105fc223b3e5aac3e20dcd026b83b38b 2019-07-04T12:41:05.743+0300 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013 2019-07-04T12:41:05.743+0300 I CONTROL [initandlisten] allocator: tcmalloc 2019-07-04T12:41:05.743+0300 I CONTROL [initandlisten] modules: none 2019-07-04T12:41:05.743+0300 I CONTROL [initandlisten] build environment: 2019-07-04T12:41:05.743+0300 I CONTROL [initandlisten] distmod: rhel62 2019-07-04T12:41:05.743+0300 I CONTROL [initandlisten] distarch: x86_64 2019-07-04T12:41:05.743+0300 I CONTROL [initandlisten] target_arch: x86_64 2019-07-04T12:41:05.744+0300 I CONTROL [initandlisten] options: \{ config: "/home/dfnuat/config/mongod.conf", net: { port: 27017 }, processManagement: \{ fork: true, pidFilePath: "/home/dfnuat/data/mongodb/mongod.pid" }, storage: \{ dbPath: "/home/dfnuat/data/mongodb", journal: { enabled: true } }, systemLog: \{ destination: "file", logAppend: true, path: "/home/dfnuat/logs/mongodb/mongod.log" } } 2019-07-04T12:41:05.744+0300 W - [initandlisten] Detected unclean shutdown - /home/dfnuat/data/mongodb/mongod.lock is not empty. 2019-07-04T12:41:05.775+0300 I - [initandlisten] Detected data files in /home/dfnuat/data/mongodb created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'. 2019-07-04T12:41:05.775+0300 W STORAGE [initandlisten] Recovering data from the last clean checkpoint. 2019-07-04T12:41:05.775+0300 I STORAGE [initandlisten] 2019-07-04T12:41:05.775+0300 I STORAGE [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine 2019-07-04T12:41:05.775+0300 I STORAGE [initandlisten] ** See http://dochub.mongodb.org/core/prodnotes-filesystem 2019-07-04T12:41:05.775+0300 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=15539M,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0), 2019-07-04T12:41:06.166+0300 I CONTROL [initandlisten] 2019-07-04T12:41:06.167+0300 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database. 2019-07-04T12:41:06.167+0300 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted. 2019-07-04T12:41:06.167+0300 I CONTROL [initandlisten] 2019-07-04T12:41:06.196+0300 I FTDC [initandlisten] Initializing full-time diagnostic data capture with directory '/home/dfnuat/data/mongodb/diagnostic.data' 2019-07-04T12:41:06.197+0300 I NETWORK [thread1] waiting for connections on port 27017 2019-07-04T12:41:07.016+0300 I FTDC [ftdc] Unclean full-time diagnostic data capture shutdown detected, found interim file, some metrics may have been lost. OK 2019-07-04T12:41:11.575+0300 I NETWORK [thread1] connection accepted from 10.160.192.63:56256 #1 (1 connection now open) 2019-07-04T12:41:11.575+0300 I NETWORK [conn1] received client metadata from 10.160.192.63:56256 conn1: \{ driver: { name: "mongo-java-driver", version: "3.4.2" }, os: \{ type: "Linux", name: "Linux", architecture: "amd64", version: "2.6.32-754.12.1.el6.x86_64" }, platform: "Java/Oracle Corporation/1.8.0_121-b13" } 2019-07-04T12:41:13.618+0300 I NETWORK [thread1] connection accepted from 10.160.192.63:56258 #2 (2 connections now open) 2019-07-04T12:41:13.618+0300 I NETWORK [conn2] received client metadata from 10.160.192.63:56258 conn2: \{ driver: { name: "mongo-java-driver", version: "3.4.2" }, os: \{ type: "Linux", name: "Linux", architecture: "amd64", version: "2.6.32-754.12.1.el6.x86_64" }, platform: "Java/Oracle Corporation/1.8.0_121-b13" } 2019-07-04T12:43:29.970+0300 I NETWORK [thread1] connection accepted from 10.160.192.95:49202 #3 (3 connections now open) 2019-07-04T12:43:29.971+0300 E - [conn3] InputStreamSecureRandom failed to generate random bytes 2019-07-04T12:43:29.971+0300 I - [conn3] Fatal Assertion 28840 at src/mongo/platform/random.cpp 167 2019-07-04T12:43:29.971+0300 I - [conn3] ***aborting after fassert() failure 2019-07-04T12:43:29.985+0300 F - [conn3] Got signal: 6 (Aborted). 0x5563705afd21 0x5563705aee19 0x5563705af2fd 0x7fc4e28787e0 0x7fc4e25074f5 0x7fc4e2508cd5 0x55636f837c9d 0x556370353dc9 0x55636fa78b96 0x55636fa964d7 0x55636fa97870 0x5563700b354d 0x55636fcb43e2 0x55636fcb6356 0x55636f8a64fd 0x55636f8a6e3d 0x556370515232 0x7fc4e2870aa1 0x7fc4e25bdc4d ----- BEGIN BACKTRACE ----- {"backtrace":[\{"b":"55636F026000","o":"1589D21","s":"_ZN5mongo15printStackTraceERSo"},\{"b":"55636F026000","o":"1588E19"},\{"b":"55636F026000","o":"15892FD"},\{"b":"7FC4E2869000","o":"F7E0"},\{"b":"7FC4E24D5000","o":"324F5","s":"gsignal"},\{"b":"7FC4E24D5000","o":"33CD5","s":"abort"},\{"b":"55636F026000","o":"811C9D","s":"_ZN5mongo32fassertFailedNoTraceWithLocationEiPKcj"},\{"b":"55636F026000","o":"132DDC9","s":"_ZN5mongo23InputStreamSecureRandom9nextInt64Ev"},\{"b":"55636F026000","o":"A52B96","s":"_ZN5mongo11CmdGetNonce3runEPNS_16OperationContextERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERNS_7BSONObjEiRS8_RNS_14BSONObjBuilderE"},\{"b":"55636F026000","o":"A704D7","s":"_ZN5mongo7Command3runEPNS_16OperationContextERKNS_3rpc16RequestInterfaceEPNS3_21ReplyBuilderInterfaceE"},\{"b":"55636F026000","o":"A71870","s":"_ZN5mongo7Command11execCommandEPNS_16OperationContextEPS0_RKNS_3rpc16RequestInterfaceEPNS4_21ReplyBuilderInterfaceE"},\{"b":"55636F026000","o":"108D54D","s":"_ZN5mongo11runCommandsEPNS_16OperationContextERKNS_3rpc16RequestInterfaceEPNS2_21ReplyBuilderInterfaceE"},\{"b":"55636F026000","o":"C8E3E2"},\{"b":"55636F026000","o":"C90356","s":"_ZN5mongo16assembleResponseEPNS_16OperationContextERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE"},\{"b":"55636F026000","o":"8804FD","s":"_ZN5mongo23ServiceEntryPointMongod12_sessionLoopERKSt10shared_ptrINS_9transport7SessionEE"},\{"b":"55636F026000","o":"880E3D"},\{"b":"55636F026000","o":"14EF232"},\{"b":"7FC4E2869000","o":"7AA1"},\{"b":"7FC4E24D5000","o":"E8C4D","s":"clone"}],"processInfo":\{ "mongodbVersion" : "3.4.2", "gitVersion" : "3f76e40c105fc223b3e5aac3e20dcd026b83b38b", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "2.6.32-754.12.1.el6.x86_64", "version" : "#1 SMP Thu Mar 7 22:07:44 EST 2019", "machine" : "x86_64" }, "somap" : [ \{ "b" : "55636F026000", "elfType" : 3, "buildId" : "0409C529A50A34D3E255B4350462A560B78F8892" }, \{ "b" : "7FFD2E49D000", "elfType" : 3, "buildId" : "5E4B2841D2BD1BAB1DF5DC73366730BCE4C83264" }, \{ "b" : "7F8E05911000", "path" : "/usr/lib64/libssl.so.10", "elfType" : 3, "buildId" : "D256E285C5E11D9A99EB04CA7651003A8F67B64E" }, \{ "b" : "7F8E09D2C000", "path" : "/usr/lib64/libcrypto.so.10", "elfType" : 3, "buildId" : "1EDB45C205A844A75EBBB4F0075E705803FFB85B" }, \{ "b" : "7F8E0E724000", "path" : "/lib64/librt.so.1", "elfType" : 3, "buildId" : "552CEC3216281CCFD7FA6432C723D50163255823" }, \{ "b" : "7F8E0ED20000", "path" : "/lib64/libdl.so.2", "elfType" : 3, "buildId" : "2AF795BFFD122309BA3359FEBABB5D0967403D17" }, \{ "b" : "7F8E0EE9C000", "path" : "/lib64/libm.so.6", "elfType" : 3, "buildId" : "4AAEE970B045D8BF946578B9C7F3AB5CDE9AB44A" }, \{ "b" : "7F8E0C886000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3, "buildId" : "EDC925E58FE28DCA536993EB13179C739F1E6566" }, \{ "b" : "7F8E0E269000", "path" : "/lib64/libpthread.so.0", "elfType" : 3, "buildId" : "4EA475CD3FD3B69B6C95D9381FA74B36DB4992EF" }, \{ "b" : "7F8E0EAD5000", "path" : "/lib64/libc.so.6", "elfType" : 3, "buildId" : "A5F93642576869A169BACF9771C674E0CB91FCBB" }, \{ "b" : "7F8E1037D000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "97AF4B77212F74CFF72B6C013E6AA2D74A97EF60" }, \{ "b" : "7F8E08091000", "path" : "/lib64/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "0C249DF4D77989253CCD859956BF50749308A16A" }, \{ "b" : "7F8E079AA000", "path" : "/lib64/libkrb5.so.3", "elfType" : 3, "buildId" : "624C7056B8BBE6BA758DEF557F516FBDBD01E1FD" }, \{ "b" : "7F8E097A6000", "path" : "/lib64/libcom_err.so.2", "elfType" : 3, "buildId" : "57F77704A7F1F4E3689D028D3F9ADD4E77486EC9" }, \{ "b" : "7F8E0897A000", "path" : "/lib64/libk5crypto.so.3", "elfType" : 3, "buildId" : "C81673692EEF670BC951EE726490F5D1CAB822F4" }, \{ "b" : "7F8E0BF60000", "path" : "/lib64/libresolv.so.2", "elfType" : 3, "buildId" : "4786A2A5D30B121601958E84D643C70C13C4FBA5" }, \{ "b" : "7F8E0C94A000", "path" : "/lib64/libz.so.1", "elfType" : 3, "buildId" : "D053BB4FF0C2FC983842F81598813B9B931AD0D1" }, \{ "b" : "7F8E0673F000", "path" : "/lib64/libkrb5support.so.0", "elfType" : 3, "buildId" : "03B69EEB8998AC9CA7519A27571BAD976BA4C56D" }, \{ "b" : "7F8E0693C000", "path" : "/lib64/libkeyutils.so.1", "elfType" : 3, "buildId" : "3BCCABE75DC61BBA81AAE45D164E26EF4F9F55DB" }, \{ "b" : "7F8E0BF1D000", "path" : "/lib64/libselinux.so.1", "elfType" : 3, "buildId" : "B4576BE308DDCF7BC31F7304E4734C3D846D0236" } ] }} mongod(_ZN5mongo15printStackTraceERSo+0x41) [0x5563705afd21] mongod(+0x1588E19) [0x5563705aee19] mongod(+0x15892FD) [0x5563705af2fd] libpthread.so.0(+0xF7E0) [0x7fc4e28787e0] libc.so.6(gsignal+0x35) [0x7fc4e25074f5] libc.so.6(abort+0x175) [0x7fc4e2508cd5] mongod(_ZN5mongo32fassertFailedNoTraceWithLocationEiPKcj+0x0) [0x55636f837c9d] mongod(_ZN5mongo23InputStreamSecureRandom9nextInt64Ev+0x109) [0x556370353dc9] mongod(_ZN5mongo11CmdGetNonce3runEPNS_16OperationContextERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERNS_7BSONObjEiRS8_RNS_14BSONObjBuilderE+0x56) [0x55636fa78b96] mongod(_ZN5mongo7Command3runEPNS_16OperationContextERKNS_3rpc16RequestInterfaceEPNS3_21ReplyBuilderInterfaceE+0x527) [0x55636fa964d7] mongod(_ZN5mongo7Command11execCommandEPNS_16OperationContextEPS0_RKNS_3rpc16RequestInterfaceEPNS4_21ReplyBuilderInterfaceE+0x9E0) [0x55636fa97870] mongod(_ZN5mongo11runCommandsEPNS_16OperationContextERKNS_3rpc16RequestInterfaceEPNS2_21ReplyBuilderInterfaceE+0x26D) [0x5563700b354d] mongod(+0xC8E3E2) [0x55636fcb43e2] mongod(_ZN5mongo16assembleResponseEPNS_16OperationContextERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x6D6) [0x55636fcb6356] mongod(_ZN5mongo23ServiceEntryPointMongod12_sessionLoopERKSt10shared_ptrINS_9transport7SessionEE+0x1ED) [0x55636f8a64fd] mongod(+0x880E3D) [0x55636f8a6e3d] mongod(+0x14EF232) [0x556370515232] libpthread.so.0(+0x7AA1) [0x7fc4e2870aa1] libc.so.6(clone+0x6D) [0x7fc4e25bdc4d] ----- END BACKTRACE ----- 2019-07-04T12:44:29.200+0300 I CONTROL [main] ***** SERVER RESTARTED ***** 2019-07-04T12:44:29.211+0300 I CONTROL [initandlisten] MongoDB starting : pid=7110 port=27017 dbpath=/home/dfnuat/data/mongodb 64-bit host=localhost 2019-07-04T12:44:29.211+0300 I CONTROL [initandlisten] db version v3.4.2 2019-07-04T12:44:29.211+0300 I CONTROL [initandlisten] git version: 3f76e40c105fc223b3e5aac3e20dcd026b83b38b 2019-07-04T12:44:29.211+0300 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013 2019-07-04T12:44:29.211+0300 I CONTROL [initandlisten] allocator: tcmalloc 2019-07-04T12:44:29.211+0300 I CONTROL [initandlisten] modules: none 2019-07-04T12:44:29.211+0300 I CONTROL [initandlisten] build environment: 2019-07-04T12:44:29.211+0300 I CONTROL [initandlisten] distmod: rhel62 2019-07-04T12:44:29.211+0300 I CONTROL [initandlisten] distarch: x86_64 2019-07-04T12:44:29.211+0300 I CONTROL [initandlisten] target_arch: x86_64 2019-07-04T12:44:29.211+0300 I CONTROL [initandlisten] options: \{ config: "/home/dfnuat/config/mongod.conf", net: { port: 27017 }, processManagement: \{ fork: true, pidFilePath: "/home/dfnuat/data/mongodb/mongod.pid" }, storage: \{ dbPath: "/home/dfnuat/data/mongodb", journal: { enabled: true } }, systemLog: \{ destination: "file", logAppend: true, path: "/home/dfnuat/logs/mongodb/mongod.log" } } 2019-07-04T12:44:29.211+0300 W - [initandlisten] Detected unclean shutdown - /home/dfnuat/data/mongodb/mongod.lock is not empty. 2019-07-04T12:44:29.232+0300 I - [initandlisten] Detected data files in /home/dfnuat/data/mongodb created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'. 2019-07-04T12:44:29.232+0300 W STORAGE [initandlisten] Recovering data from the last clean checkpoint. 2019-07-04T12:44:29.232+0300 I STORAGE [initandlisten] 2019-07-04T12:44:29.232+0300 I STORAGE [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine 2019-07-04T12:44:29.232+0300 I STORAGE [initandlisten] ** See http://dochub.mongodb.org/core/prodnotes-filesystem 2019-07-04T12:44:29.232+0300 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=15539M,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0), 2019-07-04T12:44:29.589+0300 I CONTROL [initandlisten] 2019-07-04T12:44:29.589+0300 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database. 2019-07-04T12:44:29.589+0300 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted. 2019-07-04T12:44:29.589+0300 I CONTROL [initandlisten] 2019-07-04T12:44:29.618+0300 I FTDC [initandlisten] Initializing full-time diagnostic data capture with directory '/home/dfnuat/data/mongodb/diagnostic.data' 2019-07-04T12:44:29.619+0300 I NETWORK [thread1] waiting for connections on port 27017 2019-07-04T12:44:30.026+0300 I FTDC [ftdc] Unclean full-time diagnostic data capture shutdown detected, found interim file, some metrics may have been lost. OK 2019-07-04T12:44:31.595+0300 I NETWORK [thread1] connection accepted from 10.160.192.63:56408 #1 (1 connection now open) 2019-07-04T12:44:31.596+0300 I NETWORK [conn1] received client metadata from 10.160.192.63:56408 conn1: \{ driver: { name: "mongo-java-driver", version: "3.4.2" }, os: \{ type: "Linux", name: "Linux", architecture: "amd64", version: "2.6.32-754.12.1.el6.x86_64" }, platform: "Java/Oracle Corporation/1.8.0_121-b13" } 2019-07-04T12:44:33.637+0300 I NETWORK [thread1] connection accepted from 10.160.192.63:56410 #2 (2 connections now open) 2019-07-04T12:44:33.637+0300 I NETWORK [conn2] received client metadata from 10.160.192.63:56410 conn2: \{ driver: { name: "mongo-java-driver", version: "3.4.2" }, os: \{ type: "Linux", name: "Linux", architecture: "amd64", version: "2.6.32-754.12.1.el6.x86_64" }, platform: "Java/Oracle Corporation/1.8.0_121-b13" } 2019-07-04T13:06:22.777+0300 I NETWORK [thread1] connection accepted from 10.237.149.228:49505 #3 (3 connections now open) 2019-07-04T13:06:37.684+0300 I - [conn3] end connection 10.237.149.228:49505 (3 connections now open) 2019-07-04T13:13:51.695+0300 I NETWORK [thread1] connection accepted from 10.237.149.228:50554 #4 (3 connections now open) 2019-07-04T13:13:51.698+0300 I NETWORK [thread1] connection accepted from 10.237.149.228:50555 #5 (4 connections now open) 2019-07-04T13:13:51.699+0300 I - [conn4] end connection 10.237.149.228:50554 (4 connections now open) 2019-07-04T13:13:51.915+0300 I - [conn5] end connection 10.237.149.228:50555 (3 connections now open) 2019-07-04T13:17:22.203+0300 I NETWORK [thread1] connection accepted from 10.237.149.228:50865 #6 (3 connections now open) 2019-07-04T13:17:27.355+0300 I - [conn6] end connection 10.237.149.228:50865 (3 connections now open)

Top User Comments

daniel.hatcher commented on Wed, 31 Jul 2019 13:55:11 +0000: I'm glad to hear you got around your problem. If you do upgrade to the newest version and encounter this issue again, please leave a comment on this ticket. lahirusmpfb@gmail.com commented on Wed, 31 Jul 2019 08:08:14 +0000: Dear @Daniel, So sorry for the delayed reply. Actually I have not tried to update the server version because I had already decided to get all necessary documents in different collections by running several find queries in the shell and restore the other DB by running insert/update queries. that is not the solution I must say  . anyway  thanks for the support provided. daniel.hatcher commented on Mon, 29 Jul 2019 20:53:55 +0000: lahirusamith, have you had a chance to read my previous comment? daniel.hatcher commented on Mon, 8 Jul 2019 19:54:32 +0000: We've seen this issue once or twice in the past and linked it to an OS anomaly. Do you receive a fatal assertion every time you run the mongodump? Could you try upgrading your binaries to 3.4.21 (the latest release of 3.4) and see if the problem still happens? If it still occurs post-3.4.21, could you please provide a full mongod log file as well as the exact timestamp you ran the command?

Steps to Reproduce