KEMBAR78
Parquet: add `on_bad_file` argument to error/warn/skip bad files by lhoestq · Pull Request #7806 · huggingface/datasets · GitHub
Skip to content

Parquet: add on_bad_file argument to error/warn/skip bad files #7806

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Oct 9, 2025
Merged

Parquet: add on_bad_file argument to error/warn/skip bad files #7806

merged 1 commit into from
Oct 9, 2025
+38 −12

Conversation

@lhoestq
Copy link
Member

@lhoestq lhoestq commented Oct 9, 2025 •
edited
Loading 

from datasets import load_dataset

on_bad_file = "error"  # default
# on_bad_file = "warn"  # warn and skip bad file
# on_bad_file = "skip"  # skip bad file
ds = load_dataset(parquet_dataset_id, on_bad_file=on_bad_file)

@HuggingFaceDocBuilderDev
Copy link

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@lhoestq lhoestq merged commit 5eec91a into main Oct 9, 2025
13 of 15 checks passed
@lhoestq lhoestq deleted the parquet-on_bad_file branch October 9, 2025 16:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@lhoestq @HuggingFaceDocBuilderDev