Search results

bug: o200k_harmony duplicates special token id 200018

Status: Open.

#457 In openai/tiktoken;

· jinzhuer opened on Oct 13, 2025

0.12.0 no longer provides manylinux2014 wheels and breaks builds in AWS SAM Python 3.11 images(requires Rust compiler)

Status: Open.

#456 In openai/tiktoken;

· jotasixto opened on Oct 10, 2025

Interest in porting any of the implementation from TokenDagger? (With Possible PR)

Status: Open.

#441 In openai/tiktoken;

· Jeffrharr opened on Aug 26, 2025

Unable to Release Memory Used by Tiktoken Encoding After Setting to None and GC Collect

Status: Open.

#414 In openai/tiktoken;

· kimhyun5u opened on Jul 14, 2025

dead lock

Status: Open.

#413 In openai/tiktoken;

· tiandalu opened on Jul 11, 2025

Bug in buffer protocol implementation

Status: Open.

#405 In openai/tiktoken;

· mariosasko opened on Jun 1, 2025

Fail to Install on win-arm64

Status: Open.

#403 In openai/tiktoken;

· MrHinsh opened on May 31, 2025

RuntimeError(StackOverflow) with tiktoken.encoding_for_model('gpt-4').encode(txt, disallowed_special=())

Status: Open.

#400 In openai/tiktoken;

· sumitKpandey opened on May 6, 2025

Feature Request: Integrate Token Usage Cost Calculation Based on OpenAI's Pricing

Status: Open.

#375 In openai/tiktoken;

· Programmer-RD-AI opened on Feb 2, 2025

tiktoken under counting tokens for openai text-embedding-3-large

Status: Open.

#366 In openai/tiktoken;

· mattkauffman23 opened on Jan 9, 2025

The license file is not well referenced

Status: Open.

#362 In openai/tiktoken;

· tcosta-travelperk opened on Dec 18, 2024

Community Resource: AutoTikTokenizer - A Bridge Between TikToken and HuggingFace Tokenizers

Status: Open.

#358 In openai/tiktoken;

· bhavnicksm opened on Nov 7, 2024