OPERATIONAL DEFECT DATABASE
...
...
BugZero found this defect 2809 days ago.
Hello, unexpected error occurred while trying to remove document that contains invalid geoJson from collection with partial index: WriteResult({ "nRemoved" : 0, "writeError" : { "code" : 16755, "errmsg" : "Can't extract geo keys: { _id: ObjectId('58f8e5e06758c605c09e9110'), geometry_status: -1.0, geoJson: { type: \"Polygon\", coordinates: [ [ [ 5.86121301988361, 51.9128247961245 ], [ 5.86121303441693, 51.9128247960658 ], [ 5.86121306203623, 51.9128260363524 ], [ 5.86077631455415, 51.9128299191492 ], [ 5.86076411163462, 51.9122347660331 ], [ 5.8612026892498, 51.9122313253 ], [ 5.8612136159931, 51.912764202971 ], [ 5.8613461964441, 51.9127630207468 ], [ 5.86134760592148, 51.9128235878296 ], [ 5.86121486552018, 51.9128247796886 ], [ 5.86121303441693, 51.9128247960658 ], [ 5.86121301988361, 51.9128247961245 ] ] ] } } Loop is not valid: [ [ 5.86121301988361, 51.9128247961245 ], [ 5.86121303441693, 51.9128247960658 ], [ 5.86121306203623, 51.9128260363524 ], [ 5.86077631455415, 51.9128299191492 ], [ 5.86076411163462, 51.9122347660331 ], [ 5.8612026892498, 51.9122313253 ], [ 5.8612136159931, 51.912764202971 ], [ 5.8613461964441, 51.9127630207468 ], [ 5.86134760592148, 51.9128235878296 ], [ 5.86121486552018, 51.9128247796886 ], [ 5.86121303441693, 51.9128247960658 ], [ 5.86121301988361, 51.9128247961245 ] ] Duplicate vertices: 1 and 10" } }) Partial index is 2dsphere with condition geometry_status >= 0. Inserted document has invalid goeJson but geometry_status = -1 so it can be inserted. But cannot be removed.
xgen-internal-githook commented on Thu, 8 Feb 2018 22:00:13 +0000: Author: {'email': 'louis.williams@mongodb.com', 'name': 'Louis Williams', 'username': 'louiswilliams'} Message: SERVER-28975 Fix typos Branch: master https://github.com/mongodb/mongo/commit/1e121053951d4b5dca898f4b7939d0a98d7bde09 louis.williams commented on Thu, 8 Feb 2018 21:42:58 +0000: The following commit was made to master with the wrong ticket in the commit message: Message: SERVER-29876 Check if partial filter applies when unindexing documents Branch: master https://github.com/mongodb/mongo/commit/b77e46685dc7ae8b9893ef019a1cbc67798bb749 ian@10gen.com commented on Mon, 16 Oct 2017 14:16:33 +0000: louis.williams we're putting this back on the backlog just to make sure it doesn't get lost from the Storage team's view, but please ping me or Eric if you do get spare time and come up with a patch for this. louis.williams commented on Fri, 13 Oct 2017 19:00:46 +0000: After talking with max.hirschhorn and david.storch use the kRelaxConstraints constant to ignore whitelisted errors when calling IndexAccessMethod::getKeys(). If a whitelisted error is thrown, but the partialIndexFilter applies to the document, then the error will be thrown. louis.williams commented on Thu, 12 Oct 2017 17:40:27 +0000: It seems like these are the only potential solutions. max.hirschhorn and david.storch let me know if you agree. Solution 3 is a much larger project. 1. On unindex(), check if the partialIndex filter applies to the document. If it doesn't, don't attempt to unindex it. Problem: If the semantics of filter matching change in a future release, an index could be left never deleted on removal, dangling. 2. On removal, use the existing kRelaxConstraints constant to ignore all key errors when calling IndexAccessMethod::getKeys(). Problem: An error retrieving one key for a document in the set will prevent the retrieval, and subsequent unindexing of keys for other later valid documents. 3. On remove, use the existing kRelaxConstraints, to ignore individual key errors, and pass to the IndexAccessMethod::doGetKeys() virtual function, which all subclasses must implement. This means all key retrieval errors must be handled on a document-by-document basis, and ignored explicitly when kRelaxConstraints is set. If an error is encountered, the retrieval of the key is aborted, but the next documents are still checked to ensure that no keys are skipped. milkie commented on Wed, 26 Apr 2017 12:46:39 +0000: That's a better workaround. I wasn't sure that that method would work (it may have given a similar error). Thanks for the tip! andrei.belashou commented on Wed, 26 Apr 2017 12:40:50 +0000: Hi Eric, thanks for reply. We also found out another way to remove document. It is possible to update document and set any valid indexable geoJson and then remove the document. In this case we don't need to drop and create index. Might be helpful for someone. milkie commented on Wed, 26 Apr 2017 12:35:59 +0000: Hi Andrei, Thanks for filing this bug report. We will try to fix it in a future minor release of 3.4, but in the meantime, the only way I can think of to be able to remove the document is to drop the geo index, delete the document, and then add the index back.
// Create collection with partial 2dsphere index (condition is geometry_status >= 0) db.test.createIndex( {"geoJson": "2dsphere"}, { partialFilterExpression: {"geometry_status": {$gte:0}}} ); // Insert document that doesn't meet partial index condition db.test.insert({ "_id": ObjectId('58f8e5e06758c605c09e9110'), "geometry_status" : -1, "geoJson" : { "type" : "Polygon", "coordinates" : [ [ [ 5.86121301988361, 51.9128247961245 ], [ 5.86121303441693, 51.9128247960658 ], [ 5.86121306203623, 51.9128260363524 ], [ 5.86077631455415, 51.9128299191492 ], [ 5.86076411163462, 51.9122347660331 ], [ 5.8612026892498, 51.9122313253 ], [ 5.8612136159931, 51.912764202971 ], [ 5.8613461964441, 51.9127630207468 ], [ 5.86134760592148, 51.9128235878296 ], [ 5.86121486552018, 51.9128247796886 ], [ 5.86121303441693, 51.9128247960658 ], [ 5.86121301988361, 51.9128247961245 ] ] ] }}); // Remove this document db.test.remove({"_id": ObjectId('58f8e5e06758c605c09e9110')}); // Error
