KEMBAR78
Deprecate duplicate keys in object by byroot · Pull Request #818 · ruby/json · GitHub
Skip to content

Conversation

@byroot
Copy link
Member

@byroot byroot commented Jun 23, 2025

There are few legitimate use cases for duplicate keys, and can in some case be exploited.

Rather to always silently accept them, we should emit a warning, and in the future require to explictly allow them.

TODO: jruby version

@byroot byroot force-pushed the duplicate-keys branch 2 times, most recently from 6e6bd2a to 5ad5d37 Compare June 23, 2025 15:30
There are few legitimate use cases for duplicate keys, and can
in some case be exploited.

Rather to always silently accept them, we should emit a warning,
and in the future require to explictly allow them.
@byroot byroot merged commit 904e0fd into ruby:master Jun 23, 2025
35 checks passed
@byroot byroot deleted the duplicate-keys branch June 23, 2025 16:09
@miharekar
Copy link

Is there a way to improve the debug message somewhat? It's impossible to find what's causing these in my test runs 😅

...
/Users/miharekar/.local/share/mise/installs/ruby/3.4.5/lib/ruby/gems/3.4.0/gems/json-2.13.1/lib/json/common.rb:338: warning: detected duplicate keys in JSON object. This will raise an error in json 3.0 unless enabled via `allow_duplicate_key: true` at line 1 column 1632
/Users/miharekar/.local/share/mise/installs/ruby/3.4.5/lib/ruby/gems/3.4.0/gems/json-2.13.1/lib/json/common.rb:338: warning: detected duplicate keys in JSON object. This will raise an error in json 3.0 unless enabled via `allow_duplicate_key: true` at line 1 column 10
/Users/miharekar/.local/share/mise/installs/ruby/3.4.5/lib/ruby/gems/3.4.0/gems/json-2.13.1/lib/json/common.rb:338: warning: detected duplicate keys in JSON object. This will raise an error in json 3.0 unless enabled via `allow_duplicate_key: true` at line 1 column 1402
/Users/miharekar/.local/share/mise/installs/ruby/3.4.5/lib/ruby/gems/3.4.0/gems/json-2.13.1/lib/json/common.rb:338: warning: detected duplicate keys in JSON object. This will raise an error in json 3.0 unless enabled via `allow_duplicate_key: true` at line 1 column 1632
...

byroot added a commit to byroot/json that referenced this pull request Jul 27, 2025
byroot added a commit to byroot/json that referenced this pull request Jul 27, 2025
@byroot
Copy link
Member Author

byroot commented Jul 27, 2025

Is there a way to improve the debug message somewhat?

#831

@byroot
Copy link
Member Author

byroot commented Jul 27, 2025

It's impossible to find what's causing these in my test runs

But also you can turn the warning into an error, that should give you a backtrace.

@miharekar
Copy link

you can turn the warning into an error

I assume this is coming from one of my dependencies so not that easy to do that without knowing which #parse call is throwing this.

@byroot
Copy link
Member Author

byroot commented Jul 27, 2025

Yeah, I'm currently working on getting the warning emitted at the caller. But you can do something like:

JSON.singleton_class.prepend(Module.new {
  def parse(source, opts = nil)
    opts ||= {}
    opts[:allow_duplicate_key] = false
    super
  end
})

NB: Not saying this is fine, just giving you a quick solution until this is improved.

byroot added a commit to byroot/json that referenced this pull request Jul 27, 2025
Followup: ruby#818

Now the warning should point at the `JSON.parse` caller, and not
inside the json gem itself.
@byroot
Copy link
Member Author

byroot commented Jul 27, 2025

With #832, the warning should point at the caller.

byroot added a commit to byroot/json that referenced this pull request Jul 27, 2025
Followup: ruby#818

Now the warning should point at the `JSON.parse` caller, and not
inside the json gem itself.
@byroot
Copy link
Member Author

byroot commented Jul 27, 2025

Another trick to handle/track warnings is to turn them into errors, e.g. https://github.com/rails/rails/blob/087e27d78395e630eb8fdd34a5203dc33fcde099/tools/strict_warnings.rb

@miharekar
Copy link

Awesome, #832 did the trick!

Thanks so much for this and all that you do! 🙏

matzbot pushed a commit to ruby/ruby that referenced this pull request Jul 28, 2025
matzbot pushed a commit to ruby/ruby that referenced this pull request Jul 28, 2025
Followup: ruby/json#818

Now the warning should point at the `JSON.parse` caller, and not
inside the json gem itself.

ruby/json@cd51557387
@renchap
Copy link

renchap commented Jul 28, 2025

If this can be useful to someone, I noticed that most of the time this happened in our app was caused by 1 hash with symbol keys being merged with one hash with string keys, then serialised to JSON, then parsed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants