Remove scripts altogether #7592

lhoestq · 2025-06-04T15:14:11Z

TODO:

remplace fixtures based on script with no-script fixtures
windaube

HuggingFaceDocBuilderDev · 2025-06-04T15:17:16Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

jp1924 · 2025-07-14T06:57:27Z

Hi @lhoestq,
I wanted to ask
are you planning to stop supporting dataset builds using GeneratorBasedBuilder?

If so, could you share the reason why?

lhoestq · 2025-07-15T10:23:14Z

We stopped supporting dataset scripts altogether, whether they are based on GeneratorBasedBuilder or any other builder. This means you can't load_dataset() a dataset script anymore. We did this mostly for security reasons which is blocking for many users and also impossible to build upon (e.g. the for the Dataset Viewer on HF)

jp1924 · 2025-07-16T18:59:07Z

Ah, so only the trust_remote_code feature of load_dataset is deprecated, and

from datasets import load_dataset_builder
  
builder = load_dataset_builder('cornell-movie-review-data/rotten_tomatoes')  
builder.download_and_prepare()

we can still load data using load_dataset_builder and download_and_prepare, right?
That's a relief. I thought the removal of trust_remote_code in load_dataset meant GeneratorBasedBuilder was being deprecated too, haha.
Got it, thanks for the clarification!

autosquid · 2025-08-04T15:14:06Z

Can you give an example on how to upgrade from using trust_remote_code? I used to load_dataset from a script generating my training data in a streaming way.

autosquid · 2025-08-04T15:16:46Z

For guys who dislike this change +1

…face/datasets#7592; changed MMLU to an up-to-date Parquet dataset repository and made the required changes to the module

lhoestq · 2025-09-04T11:03:43Z

Can you give an example on how to upgrade from using trust_remote_code? I used to load_dataset from a script generating my training data in a streaming way.

here is a command to convert a script based dataset on HF to a Parquet dataset:

uv run --with "datasets==3.6.0" datasets-cli convert_to_parquet <username/dataset-name> --trust_remote_code

And if you want to define a streaming pipeline, feel free to use the IterableDataset API instead:

from datasets import load_dataset

ds = load_dataset(..., streaming=True)  # source data
ds = ds.map(...)  # preprocessing
ds = ds.filter(...)  # remove bad samples

# For training
# data_loader = DataLoader(ds, ...)

# Saving the dataset
# ds.push_to_hub(...)

## Why are these changes needed? Hugging Face Datasets v4.0.0 removed dataset scripts and the related “dynamic modules” mechanism (See huggingface/datasets#7592), so `datasets.load.init_dynamic_modules` no longer exists. As `ray.data._internal.datasource.huggingface_datasource` currently imports/calls this function unconditionally, it raises: ``` AttributeError: module 'datasets.load' has no attribute 'init_dynamic_modules' ``` With this PR, Ray detects the legacy initializer and only calls it on Datasets ≤3.x. On Datasets ≥4.0.0 the block becomes a no-op. The behavior for older Datasets remains unchanged. ## Related issue number NA ## Checks - [x] I've signed off every commit(by using the -s flag, i.e., `git commit -s`) in this PR. - [x] I've run `scripts/format.sh` to lint the changes in this PR. - [ ] I've included any doc changes needed for https://docs.ray.io/en/master/. - [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in `doc/source/tune/api/` under the corresponding `.rst` file. - [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/ - Testing Strategy - [ ] Unit tests - [ ] Release tests - [ ] This PR is not tested :( --------- Signed-off-by: Masahiro Tanaka <mtanaka@anyscale.com> Signed-off-by: Masahiro Tanaka <81312776+tohtana@users.noreply.github.com> Signed-off-by: Lonnie Liu <lonnie@anyscale.com> Signed-off-by: Rueian <rueian@anyscale.com> Signed-off-by: Rueian <rueiancsie@gmail.com> Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com> Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com> Signed-off-by: Matthew Owen <mowen@anyscale.com> Signed-off-by: Sagar Sumit <sagarsumit09@gmail.com> Signed-off-by: xgui <xgui@anyscale.com> Signed-off-by: Xinyuan <43737116+xinyuangui2@users.noreply.github.com> Signed-off-by: Kai-Hsun Chen <kaihsun@anyscale.com> Signed-off-by: Matthew Deng <matt@anyscale.com> Signed-off-by: Goutam V <goutam@anyscale.com> Signed-off-by: root <vtsang@amd.com> Signed-off-by: Rui Qiao <ruisearch42@gmail.com> Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com> Signed-off-by: Yiwen Xiang <xyiwen@google.com> Signed-off-by: Mao Yancan <yancan.mao@bytedance.com> Signed-off-by: Mao Yancan <maoyancan@u.nus.edu> Signed-off-by: Cuong Nguyen <can@anyscale.com> Signed-off-by: iamjustinhsu <jhsu@anyscale.com> Signed-off-by: iamjustinhsu <140442892+iamjustinhsu@users.noreply.github.com> Signed-off-by: dragongu <andrewgu@vip.qq.com> Signed-off-by: Andrew Grosser <dioptre@gmail.com> Signed-off-by: Justin Yu <justinvyu@anyscale.com> Signed-off-by: abrar <abrar@anyscale.com> Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com> Signed-off-by: Yicheng-Lu-llll <luyc58576@gmail.com> Signed-off-by: Potato <tanxinyu@apache.org> Signed-off-by: Zac Policzer <zac@anyscale.com> Signed-off-by: Cuong Nguyen <128072568+can-anyscale@users.noreply.github.com> Signed-off-by: Alexey Kudinkin <ak@anyscale.com> Signed-off-by: Seiji Eicher <seiji@anyscale.com> Signed-off-by: 杨睿 <595403043@qq.com> Signed-off-by: dayshah <dhyey2019@gmail.com> Signed-off-by: axreldable <aleksei.starikov.ax@gmail.com> Signed-off-by: kaihsun <kaihsun@anyscale.com> Signed-off-by: Kai-Hsun Chen <kaihsun@apache.org> Signed-off-by: Timothy Seah <tseah@anyscale.com> Signed-off-by: kevin <kevin@anyscale.com> Signed-off-by: joshlee <joshlee@anyscale.com> Signed-off-by: 400Ping <fourhundredping@gmail.com> Signed-off-by: Dhyey Shah <dhyey2019@gmail.com> Signed-off-by: harshit <harshit@anyscale.com> Signed-off-by: anmol <anmol@anyscale.com> Signed-off-by: Kit Lee <7000003+wingkitlee0@users.noreply.github.com> Signed-off-by: JasonLi1909 <jasli1909@gmail.com> Signed-off-by: Jason Li <57246540+JasonLi1909@users.noreply.github.com> Signed-off-by: MengqingCao <cmq0113@163.com> Signed-off-by: sampan <sampan@anyscale.com> Signed-off-by: zac <zac@anyscale.com> Signed-off-by: Elliot Barnwell <elliot.barnwell@anyscale.com> Signed-off-by: Mengjin Yan <mengjinyan3@gmail.com> Signed-off-by: myan <myan@anyscale.com> Signed-off-by: Seiji Eicher <58963096+eicherseiji@users.noreply.github.com> Signed-off-by: Linkun <github@lkchen.net> Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu> Signed-off-by: Markus <msobkowski@hercules.ai> Signed-off-by: Gagandeep Singh <gdp.1807@gmail.com> Signed-off-by: akyang-anyscale <alexyang@anyscale.com> Signed-off-by: Alan Guo <aguo@anyscale.com> Signed-off-by: haotian <haotian@ebay.com> Signed-off-by: Howie Tien <sword865@163.com> Signed-off-by: will.lin <will.lin@anyscale.com> Signed-off-by: Richard Liaw <rliaw@berkeley.edu> Signed-off-by: Ryan O'Leary <ryanaoleary@google.com> Signed-off-by: Andrew Sy Kim <andrewsy@google.com> Signed-off-by: Matvei Pashkovskii <mpashkov@amd.com> Signed-off-by: Kishanthan Thangarajah <kshanth2101@gmail.com> Signed-off-by: my-vegetable-has-exploded <wy1109468038@gmail.com> Signed-off-by: Neil Girdhar <mistersheik@gmail.com> Signed-off-by: Nikhil Ghosh <nikhil@anyscale.com> Signed-off-by: win5923 <ken89@kimo.com> Signed-off-by: Stephanie wang <smwang@cs.washington.edu> Signed-off-by: Stephanie Wang <smwang@cs.washington.edu> Signed-off-by: cong.qian <cong.qian@anyscale.com> Signed-off-by: Lonnie Liu <95255098+aslonnie@users.noreply.github.com> Signed-off-by: doyoung <doyoung@anyscale.com> Signed-off-by: Doyoung Kim <34902420+landscapepainter@users.noreply.github.com> Signed-off-by: simonsays1980 <simon.zehnder@gmail.com> Signed-off-by: Rui Qiao <161574667+ruisearch42@users.noreply.github.com> Signed-off-by: Sampan S Nayak <sampansnayak2@gmail.com> Signed-off-by: vincenthhan <vincenthhan@tencent.com> Signed-off-by: jeffreyjeffreywang <jeffjeffreywang@gmail.com> Signed-off-by: irabbani <irabbani@anyscale.com> Signed-off-by: Ibrahim Rabbani <israbbani@gmail.com> Signed-off-by: avigyabb <avigyabb@stanford.edu> Signed-off-by: avibasnet31 <avigyabb@anyscale.com> Signed-off-by: Tanner Wood <tanwood@amazon.com> Signed-off-by: avigyabb <98926738+avigyabb@users.noreply.github.com> Signed-off-by: Ricardo Decal <rdecal@anyscale.com> Signed-off-by: Matthew <mweston3@illinois.edu> Signed-off-by: tianyi-ge <tianyig@outlook.com> Signed-off-by: can <can@anyscale.com> Signed-off-by: Alexey Kudinkin <alexey.kudinkin@gmail.com> Signed-off-by: ahao-anyscale <ahao@anyscale.com> Signed-off-by: Lehui Liu <lehui@anyscale.com> Signed-off-by: Mark Rossetti <marosset@microsoft.com> Signed-off-by: hejialing.hjl <hejialing.hjl@bytedance.com> Signed-off-by: qiwenju <qiwenju@xiaomi.com> Signed-off-by: Q文举 <39372502+orangeQWJ@users.noreply.github.com> Signed-off-by: ljstrnadiii <ljstrnadiii@gmail.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Lonnie Liu <95255098+aslonnie@users.noreply.github.com> Co-authored-by: Rueian <rueian@anyscale.com> Co-authored-by: Dhyey Shah <dhyey2019@gmail.com> Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com> Co-authored-by: Jiajun Yao <jeromeyjj@gmail.com> Co-authored-by: Matthew Owen <mowen@anyscale.com> Co-authored-by: Sagar Sumit <sagarsumit09@gmail.com> Co-authored-by: Xinyuan <43737116+xinyuangui2@users.noreply.github.com> Co-authored-by: Justin Yu <justinvyu@anyscale.com> Co-authored-by: Kai-Hsun Chen <kaihsun@anyscale.com> Co-authored-by: matthewdeng <matt@anyscale.com> Co-authored-by: goutamvenkat-anyscale <goutam@anyscale.com> Co-authored-by: vickytsang <vtsang@amd.com> Co-authored-by: Rui Qiao <161574667+ruisearch42@users.noreply.github.com> Co-authored-by: Elliot Barnwell <elliot.barnwell@anyscale.com> Co-authored-by: Yevet <xyiwen@google.com> Co-authored-by: Mao Yancan <maoyancan@u.nus.edu> Co-authored-by: Mao Yancan <yancan.mao@bytedance.com> Co-authored-by: Cuong Nguyen <128072568+can-anyscale@users.noreply.github.com> Co-authored-by: Srinath Krishnamachari <68668616+srinathk10@users.noreply.github.com> Co-authored-by: iamjustinhsu <140442892+iamjustinhsu@users.noreply.github.com> Co-authored-by: Alexey Kudinkin <alexey.kudinkin@gmail.com> Co-authored-by: dragongu <38997200+dragongu@users.noreply.github.com> Co-authored-by: Andrew Grosser <dioptre@gmail.com> Co-authored-by: Abrar Sheikh <abrar@anyscale.com> Co-authored-by: kourosh hakhamaneshi <31483498+kouroshHakha@users.noreply.github.com> Co-authored-by: Yicheng-Lu-llll <51814063+Yicheng-Lu-llll@users.noreply.github.com> Co-authored-by: Potato <tanxinyu@apache.org> Co-authored-by: Zac Policzer <zac@anyscale.com> Co-authored-by: Alexey Kudinkin <ak@anyscale.com> Co-authored-by: Seiji Eicher <58963096+eicherseiji@users.noreply.github.com> Co-authored-by: 杨睿 <595403043@qq.com> Co-authored-by: Ibrahim Rabbani <israbbani@gmail.com> Co-authored-by: Aleksei Starikov <aleksei.starikov.ax@gmail.com> Co-authored-by: Stephanie Wang <swang@cs.berkeley.edu> Co-authored-by: Timothy Seah <timothy.seah777@yahoo.com> Co-authored-by: Timothy Seah <tseah@anyscale.com> Co-authored-by: Kevin H. Luu <kevin@anyscale.com> Co-authored-by: Qiaolin Yu <liin1211@outlook.com> Co-authored-by: Joshua Lee <73967497+Sparks0219@users.noreply.github.com> Co-authored-by: Ping <fourhundredping@gmail.com> Co-authored-by: harshit-anyscale <harshit@anyscale.com> Co-authored-by: Anmol Singh <kshasingh@gmail.com> Co-authored-by: anmol <anmol@anyscale.com> Co-authored-by: Kit Lee <7000003+wingkitlee0@users.noreply.github.com> Co-authored-by: Jason Li <57246540+JasonLi1909@users.noreply.github.com> Co-authored-by: matthewdeng <matthew.j.deng@gmail.com> Co-authored-by: Mengqing Cao <cmq0113@163.com> Co-authored-by: Sampan S Nayak <sampansnayak2@gmail.com> Co-authored-by: sampan <sampan@anyscale.com> Co-authored-by: Sven Mika <svenmika1977@gmail.com> Co-authored-by: Mengjin Yan <mengjinyan3@gmail.com> Co-authored-by: Nary Yeh <60069744+machichima@users.noreply.github.com> Co-authored-by: lkchen <github@lkchen.net> Co-authored-by: Balaji Veeramani <balaji@anyscale.com> Co-authored-by: Markus <44006014+minosvasilias@users.noreply.github.com> Co-authored-by: czgdp1807 <gdp.1807@gmail.com> Co-authored-by: akyang-anyscale <alexyang@anyscale.com> Co-authored-by: Alan Guo <aguo@anyscale.com> Co-authored-by: Howie Tien <haotian@ebay.com> Co-authored-by: Balaji Veeramani <bveeramani@berkeley.edu> Co-authored-by: simonsays1980 <simon.zehnder@gmail.com> Co-authored-by: William Lin <SolitaryThinker@users.noreply.github.com> Co-authored-by: Richard Liaw <rliaw@berkeley.edu> Co-authored-by: Ryan O'Leary <113500783+ryanaoleary@users.noreply.github.com> Co-authored-by: Andrew Sy Kim <andrewsy@google.com> Co-authored-by: Matvei Pashkovskii <matvei.pashkovskii@amd.com> Co-authored-by: Kourosh Hakhamaneshi <kourosh@anyscale.com> Co-authored-by: Kishanthan Thangarajah <kshanth2101@gmail.com> Co-authored-by: yi wang <48236141+my-vegetable-has-exploded@users.noreply.github.com> Co-authored-by: Neil Girdhar <mistersheik@gmail.com> Co-authored-by: Nikhil G <nrghosh@users.noreply.github.com> Co-authored-by: Jun-Hao Wan <ken89@kimo.com> Co-authored-by: Kai-Hsun Chen <kaihsun@apache.org> Co-authored-by: Stephanie Wang <smwang@cs.washington.edu> Co-authored-by: coqian <cong.qian@anyscale.com> Co-authored-by: angelinalg <122562471+angelinalg@users.noreply.github.com> Co-authored-by: Doyoung Kim <34902420+landscapepainter@users.noreply.github.com> Co-authored-by: vincenthhan <46981434+BestVIncent@users.noreply.github.com> Co-authored-by: vincenthhan <vincenthhan@tencent.com> Co-authored-by: Jeffrey Wang <jeffrey31415926@gmail.com> Co-authored-by: jeffreyjeffreywang <jeffjeffreywang@gmail.com> Co-authored-by: Ibrahim Rabbani <irabbani@anyscale.com> Co-authored-by: avigyabb <98926738+avigyabb@users.noreply.github.com> Co-authored-by: avibasnet31 <avigyabb@anyscale.com> Co-authored-by: tannerdwood <71387269+tannerdwood@users.noreply.github.com> Co-authored-by: Tanner Wood <tanwood@amazon.com> Co-authored-by: Ricardo Decal <crypdick@users.noreply.github.com> Co-authored-by: Kamil Kaczmarek <kaczmarek.poczta@gmail.com> Co-authored-by: Hassam Ullah Sheikh <hassamsheikh1@gmail.com> Co-authored-by: MatthewCWeston <61944935+MatthewCWeston@users.noreply.github.com> Co-authored-by: Artur Niederfahrenhorst <attaismyname@googlemail.com> Co-authored-by: Artur Niederfahrenhorst <artur@anyscale.com> Co-authored-by: Tianyi <tianyig@outlook.com> Co-authored-by: gangsf <gang@anyscale.com> Co-authored-by: Gang Zhao <gang@gang-JQ62HD2C37.local> Co-authored-by: ahao-anyscale <ahao@anyscale.com> Co-authored-by: Lehui Liu <lehui@anyscale.com> Co-authored-by: Mark Rossetti <marosset@microsoft.com> Co-authored-by: Jialing He <burglarralgrub@gmail.com> Co-authored-by: hejialing.hjl <hejialing.hjl@bytedance.com> Co-authored-by: Cuong Nguyen <can@anyscale.com> Co-authored-by: Q文举 <39372502+orangeQWJ@users.noreply.github.com> Co-authored-by: qiwenju <qiwenju@xiaomi.com> Co-authored-by: Len Strnad <ljstrnadiii@gmail.com> Co-authored-by: Goku Mohandas <gokumd@gmail.com>

…oject#55908) ## Why are these changes needed? Hugging Face Datasets v4.0.0 removed dataset scripts and the related “dynamic modules” mechanism (See huggingface/datasets#7592), so `datasets.load.init_dynamic_modules` no longer exists. As `ray.data._internal.datasource.huggingface_datasource` currently imports/calls this function unconditionally, it raises: ``` AttributeError: module 'datasets.load' has no attribute 'init_dynamic_modules' ``` With this PR, Ray detects the legacy initializer and only calls it on Datasets ≤3.x. On Datasets ≥4.0.0 the block becomes a no-op. The behavior for older Datasets remains unchanged. ## Related issue number NA ## Checks - [x] I've signed off every commit(by using the -s flag, i.e., `git commit -s`) in this PR. - [x] I've run `scripts/format.sh` to lint the changes in this PR. - [ ] I've included any doc changes needed for https://docs.ray.io/en/master/. - [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in `doc/source/tune/api/` under the corresponding `.rst` file. - [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/ - Testing Strategy - [ ] Unit tests - [ ] Release tests - [ ] This PR is not tested :( --------- Signed-off-by: Masahiro Tanaka <mtanaka@anyscale.com> Signed-off-by: Masahiro Tanaka <81312776+tohtana@users.noreply.github.com> Signed-off-by: Lonnie Liu <lonnie@anyscale.com> Signed-off-by: Rueian <rueian@anyscale.com> Signed-off-by: Rueian <rueiancsie@gmail.com> Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com> Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com> Signed-off-by: Matthew Owen <mowen@anyscale.com> Signed-off-by: Sagar Sumit <sagarsumit09@gmail.com> Signed-off-by: xgui <xgui@anyscale.com> Signed-off-by: Xinyuan <43737116+xinyuangui2@users.noreply.github.com> Signed-off-by: Kai-Hsun Chen <kaihsun@anyscale.com> Signed-off-by: Matthew Deng <matt@anyscale.com> Signed-off-by: Goutam V <goutam@anyscale.com> Signed-off-by: root <vtsang@amd.com> Signed-off-by: Rui Qiao <ruisearch42@gmail.com> Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com> Signed-off-by: Yiwen Xiang <xyiwen@google.com> Signed-off-by: Mao Yancan <yancan.mao@bytedance.com> Signed-off-by: Mao Yancan <maoyancan@u.nus.edu> Signed-off-by: Cuong Nguyen <can@anyscale.com> Signed-off-by: iamjustinhsu <jhsu@anyscale.com> Signed-off-by: iamjustinhsu <140442892+iamjustinhsu@users.noreply.github.com> Signed-off-by: dragongu <andrewgu@vip.qq.com> Signed-off-by: Andrew Grosser <dioptre@gmail.com> Signed-off-by: Justin Yu <justinvyu@anyscale.com> Signed-off-by: abrar <abrar@anyscale.com> Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com> Signed-off-by: Yicheng-Lu-llll <luyc58576@gmail.com> Signed-off-by: Potato <tanxinyu@apache.org> Signed-off-by: Zac Policzer <zac@anyscale.com> Signed-off-by: Cuong Nguyen <128072568+can-anyscale@users.noreply.github.com> Signed-off-by: Alexey Kudinkin <ak@anyscale.com> Signed-off-by: Seiji Eicher <seiji@anyscale.com> Signed-off-by: 杨睿 <595403043@qq.com> Signed-off-by: dayshah <dhyey2019@gmail.com> Signed-off-by: axreldable <aleksei.starikov.ax@gmail.com> Signed-off-by: kaihsun <kaihsun@anyscale.com> Signed-off-by: Kai-Hsun Chen <kaihsun@apache.org> Signed-off-by: Timothy Seah <tseah@anyscale.com> Signed-off-by: kevin <kevin@anyscale.com> Signed-off-by: joshlee <joshlee@anyscale.com> Signed-off-by: 400Ping <fourhundredping@gmail.com> Signed-off-by: Dhyey Shah <dhyey2019@gmail.com> Signed-off-by: harshit <harshit@anyscale.com> Signed-off-by: anmol <anmol@anyscale.com> Signed-off-by: Kit Lee <7000003+wingkitlee0@users.noreply.github.com> Signed-off-by: JasonLi1909 <jasli1909@gmail.com> Signed-off-by: Jason Li <57246540+JasonLi1909@users.noreply.github.com> Signed-off-by: MengqingCao <cmq0113@163.com> Signed-off-by: sampan <sampan@anyscale.com> Signed-off-by: zac <zac@anyscale.com> Signed-off-by: Elliot Barnwell <elliot.barnwell@anyscale.com> Signed-off-by: Mengjin Yan <mengjinyan3@gmail.com> Signed-off-by: myan <myan@anyscale.com> Signed-off-by: Seiji Eicher <58963096+eicherseiji@users.noreply.github.com> Signed-off-by: Linkun <github@lkchen.net> Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu> Signed-off-by: Markus <msobkowski@hercules.ai> Signed-off-by: Gagandeep Singh <gdp.1807@gmail.com> Signed-off-by: akyang-anyscale <alexyang@anyscale.com> Signed-off-by: Alan Guo <aguo@anyscale.com> Signed-off-by: haotian <haotian@ebay.com> Signed-off-by: Howie Tien <sword865@163.com> Signed-off-by: will.lin <will.lin@anyscale.com> Signed-off-by: Richard Liaw <rliaw@berkeley.edu> Signed-off-by: Ryan O'Leary <ryanaoleary@google.com> Signed-off-by: Andrew Sy Kim <andrewsy@google.com> Signed-off-by: Matvei Pashkovskii <mpashkov@amd.com> Signed-off-by: Kishanthan Thangarajah <kshanth2101@gmail.com> Signed-off-by: my-vegetable-has-exploded <wy1109468038@gmail.com> Signed-off-by: Neil Girdhar <mistersheik@gmail.com> Signed-off-by: Nikhil Ghosh <nikhil@anyscale.com> Signed-off-by: win5923 <ken89@kimo.com> Signed-off-by: Stephanie wang <smwang@cs.washington.edu> Signed-off-by: Stephanie Wang <smwang@cs.washington.edu> Signed-off-by: cong.qian <cong.qian@anyscale.com> Signed-off-by: Lonnie Liu <95255098+aslonnie@users.noreply.github.com> Signed-off-by: doyoung <doyoung@anyscale.com> Signed-off-by: Doyoung Kim <34902420+landscapepainter@users.noreply.github.com> Signed-off-by: simonsays1980 <simon.zehnder@gmail.com> Signed-off-by: Rui Qiao <161574667+ruisearch42@users.noreply.github.com> Signed-off-by: Sampan S Nayak <sampansnayak2@gmail.com> Signed-off-by: vincenthhan <vincenthhan@tencent.com> Signed-off-by: jeffreyjeffreywang <jeffjeffreywang@gmail.com> Signed-off-by: irabbani <irabbani@anyscale.com> Signed-off-by: Ibrahim Rabbani <israbbani@gmail.com> Signed-off-by: avigyabb <avigyabb@stanford.edu> Signed-off-by: avibasnet31 <avigyabb@anyscale.com> Signed-off-by: Tanner Wood <tanwood@amazon.com> Signed-off-by: avigyabb <98926738+avigyabb@users.noreply.github.com> Signed-off-by: Ricardo Decal <rdecal@anyscale.com> Signed-off-by: Matthew <mweston3@illinois.edu> Signed-off-by: tianyi-ge <tianyig@outlook.com> Signed-off-by: can <can@anyscale.com> Signed-off-by: Alexey Kudinkin <alexey.kudinkin@gmail.com> Signed-off-by: ahao-anyscale <ahao@anyscale.com> Signed-off-by: Lehui Liu <lehui@anyscale.com> Signed-off-by: Mark Rossetti <marosset@microsoft.com> Signed-off-by: hejialing.hjl <hejialing.hjl@bytedance.com> Signed-off-by: qiwenju <qiwenju@xiaomi.com> Signed-off-by: Q文举 <39372502+orangeQWJ@users.noreply.github.com> Signed-off-by: ljstrnadiii <ljstrnadiii@gmail.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Lonnie Liu <95255098+aslonnie@users.noreply.github.com> Co-authored-by: Rueian <rueian@anyscale.com> Co-authored-by: Dhyey Shah <dhyey2019@gmail.com> Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com> Co-authored-by: Jiajun Yao <jeromeyjj@gmail.com> Co-authored-by: Matthew Owen <mowen@anyscale.com> Co-authored-by: Sagar Sumit <sagarsumit09@gmail.com> Co-authored-by: Xinyuan <43737116+xinyuangui2@users.noreply.github.com> Co-authored-by: Justin Yu <justinvyu@anyscale.com> Co-authored-by: Kai-Hsun Chen <kaihsun@anyscale.com> Co-authored-by: matthewdeng <matt@anyscale.com> Co-authored-by: goutamvenkat-anyscale <goutam@anyscale.com> Co-authored-by: vickytsang <vtsang@amd.com> Co-authored-by: Rui Qiao <161574667+ruisearch42@users.noreply.github.com> Co-authored-by: Elliot Barnwell <elliot.barnwell@anyscale.com> Co-authored-by: Yevet <xyiwen@google.com> Co-authored-by: Mao Yancan <maoyancan@u.nus.edu> Co-authored-by: Mao Yancan <yancan.mao@bytedance.com> Co-authored-by: Cuong Nguyen <128072568+can-anyscale@users.noreply.github.com> Co-authored-by: Srinath Krishnamachari <68668616+srinathk10@users.noreply.github.com> Co-authored-by: iamjustinhsu <140442892+iamjustinhsu@users.noreply.github.com> Co-authored-by: Alexey Kudinkin <alexey.kudinkin@gmail.com> Co-authored-by: dragongu <38997200+dragongu@users.noreply.github.com> Co-authored-by: Andrew Grosser <dioptre@gmail.com> Co-authored-by: Abrar Sheikh <abrar@anyscale.com> Co-authored-by: kourosh hakhamaneshi <31483498+kouroshHakha@users.noreply.github.com> Co-authored-by: Yicheng-Lu-llll <51814063+Yicheng-Lu-llll@users.noreply.github.com> Co-authored-by: Potato <tanxinyu@apache.org> Co-authored-by: Zac Policzer <zac@anyscale.com> Co-authored-by: Alexey Kudinkin <ak@anyscale.com> Co-authored-by: Seiji Eicher <58963096+eicherseiji@users.noreply.github.com> Co-authored-by: 杨睿 <595403043@qq.com> Co-authored-by: Ibrahim Rabbani <israbbani@gmail.com> Co-authored-by: Aleksei Starikov <aleksei.starikov.ax@gmail.com> Co-authored-by: Stephanie Wang <swang@cs.berkeley.edu> Co-authored-by: Timothy Seah <timothy.seah777@yahoo.com> Co-authored-by: Timothy Seah <tseah@anyscale.com> Co-authored-by: Kevin H. Luu <kevin@anyscale.com> Co-authored-by: Qiaolin Yu <liin1211@outlook.com> Co-authored-by: Joshua Lee <73967497+Sparks0219@users.noreply.github.com> Co-authored-by: Ping <fourhundredping@gmail.com> Co-authored-by: harshit-anyscale <harshit@anyscale.com> Co-authored-by: Anmol Singh <kshasingh@gmail.com> Co-authored-by: anmol <anmol@anyscale.com> Co-authored-by: Kit Lee <7000003+wingkitlee0@users.noreply.github.com> Co-authored-by: Jason Li <57246540+JasonLi1909@users.noreply.github.com> Co-authored-by: matthewdeng <matthew.j.deng@gmail.com> Co-authored-by: Mengqing Cao <cmq0113@163.com> Co-authored-by: Sampan S Nayak <sampansnayak2@gmail.com> Co-authored-by: sampan <sampan@anyscale.com> Co-authored-by: Sven Mika <svenmika1977@gmail.com> Co-authored-by: Mengjin Yan <mengjinyan3@gmail.com> Co-authored-by: Nary Yeh <60069744+machichima@users.noreply.github.com> Co-authored-by: lkchen <github@lkchen.net> Co-authored-by: Balaji Veeramani <balaji@anyscale.com> Co-authored-by: Markus <44006014+minosvasilias@users.noreply.github.com> Co-authored-by: czgdp1807 <gdp.1807@gmail.com> Co-authored-by: akyang-anyscale <alexyang@anyscale.com> Co-authored-by: Alan Guo <aguo@anyscale.com> Co-authored-by: Howie Tien <haotian@ebay.com> Co-authored-by: Balaji Veeramani <bveeramani@berkeley.edu> Co-authored-by: simonsays1980 <simon.zehnder@gmail.com> Co-authored-by: William Lin <SolitaryThinker@users.noreply.github.com> Co-authored-by: Richard Liaw <rliaw@berkeley.edu> Co-authored-by: Ryan O'Leary <113500783+ryanaoleary@users.noreply.github.com> Co-authored-by: Andrew Sy Kim <andrewsy@google.com> Co-authored-by: Matvei Pashkovskii <matvei.pashkovskii@amd.com> Co-authored-by: Kourosh Hakhamaneshi <kourosh@anyscale.com> Co-authored-by: Kishanthan Thangarajah <kshanth2101@gmail.com> Co-authored-by: yi wang <48236141+my-vegetable-has-exploded@users.noreply.github.com> Co-authored-by: Neil Girdhar <mistersheik@gmail.com> Co-authored-by: Nikhil G <nrghosh@users.noreply.github.com> Co-authored-by: Jun-Hao Wan <ken89@kimo.com> Co-authored-by: Kai-Hsun Chen <kaihsun@apache.org> Co-authored-by: Stephanie Wang <smwang@cs.washington.edu> Co-authored-by: coqian <cong.qian@anyscale.com> Co-authored-by: angelinalg <122562471+angelinalg@users.noreply.github.com> Co-authored-by: Doyoung Kim <34902420+landscapepainter@users.noreply.github.com> Co-authored-by: vincenthhan <46981434+BestVIncent@users.noreply.github.com> Co-authored-by: vincenthhan <vincenthhan@tencent.com> Co-authored-by: Jeffrey Wang <jeffrey31415926@gmail.com> Co-authored-by: jeffreyjeffreywang <jeffjeffreywang@gmail.com> Co-authored-by: Ibrahim Rabbani <irabbani@anyscale.com> Co-authored-by: avigyabb <98926738+avigyabb@users.noreply.github.com> Co-authored-by: avibasnet31 <avigyabb@anyscale.com> Co-authored-by: tannerdwood <71387269+tannerdwood@users.noreply.github.com> Co-authored-by: Tanner Wood <tanwood@amazon.com> Co-authored-by: Ricardo Decal <crypdick@users.noreply.github.com> Co-authored-by: Kamil Kaczmarek <kaczmarek.poczta@gmail.com> Co-authored-by: Hassam Ullah Sheikh <hassamsheikh1@gmail.com> Co-authored-by: MatthewCWeston <61944935+MatthewCWeston@users.noreply.github.com> Co-authored-by: Artur Niederfahrenhorst <attaismyname@googlemail.com> Co-authored-by: Artur Niederfahrenhorst <artur@anyscale.com> Co-authored-by: Tianyi <tianyig@outlook.com> Co-authored-by: gangsf <gang@anyscale.com> Co-authored-by: Gang Zhao <gang@gang-JQ62HD2C37.local> Co-authored-by: ahao-anyscale <ahao@anyscale.com> Co-authored-by: Lehui Liu <lehui@anyscale.com> Co-authored-by: Mark Rossetti <marosset@microsoft.com> Co-authored-by: Jialing He <burglarralgrub@gmail.com> Co-authored-by: hejialing.hjl <hejialing.hjl@bytedance.com> Co-authored-by: Cuong Nguyen <can@anyscale.com> Co-authored-by: Q文举 <39372502+orangeQWJ@users.noreply.github.com> Co-authored-by: qiwenju <qiwenju@xiaomi.com> Co-authored-by: Len Strnad <ljstrnadiii@gmail.com> Co-authored-by: Goku Mohandas <gokumd@gmail.com> Signed-off-by: zac <zac@anyscale.com>

…oject#55908) ## Why are these changes needed? Hugging Face Datasets v4.0.0 removed dataset scripts and the related “dynamic modules” mechanism (See huggingface/datasets#7592), so `datasets.load.init_dynamic_modules` no longer exists. As `ray.data._internal.datasource.huggingface_datasource` currently imports/calls this function unconditionally, it raises: ``` AttributeError: module 'datasets.load' has no attribute 'init_dynamic_modules' ``` With this PR, Ray detects the legacy initializer and only calls it on Datasets ≤3.x. On Datasets ≥4.0.0 the block becomes a no-op. The behavior for older Datasets remains unchanged. ## Related issue number NA ## Checks - [x] I've signed off every commit(by using the -s flag, i.e., `git commit -s`) in this PR. - [x] I've run `scripts/format.sh` to lint the changes in this PR. - [ ] I've included any doc changes needed for https://docs.ray.io/en/master/. - [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in `doc/source/tune/api/` under the corresponding `.rst` file. - [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/ - Testing Strategy - [ ] Unit tests - [ ] Release tests - [ ] This PR is not tested :( --------- Signed-off-by: Masahiro Tanaka <mtanaka@anyscale.com> Signed-off-by: Masahiro Tanaka <81312776+tohtana@users.noreply.github.com> Signed-off-by: Lonnie Liu <lonnie@anyscale.com> Signed-off-by: Rueian <rueian@anyscale.com> Signed-off-by: Rueian <rueiancsie@gmail.com> Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com> Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com> Signed-off-by: Matthew Owen <mowen@anyscale.com> Signed-off-by: Sagar Sumit <sagarsumit09@gmail.com> Signed-off-by: xgui <xgui@anyscale.com> Signed-off-by: Xinyuan <43737116+xinyuangui2@users.noreply.github.com> Signed-off-by: Kai-Hsun Chen <kaihsun@anyscale.com> Signed-off-by: Matthew Deng <matt@anyscale.com> Signed-off-by: Goutam V <goutam@anyscale.com> Signed-off-by: root <vtsang@amd.com> Signed-off-by: Rui Qiao <ruisearch42@gmail.com> Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com> Signed-off-by: Yiwen Xiang <xyiwen@google.com> Signed-off-by: Mao Yancan <yancan.mao@bytedance.com> Signed-off-by: Mao Yancan <maoyancan@u.nus.edu> Signed-off-by: Cuong Nguyen <can@anyscale.com> Signed-off-by: iamjustinhsu <jhsu@anyscale.com> Signed-off-by: iamjustinhsu <140442892+iamjustinhsu@users.noreply.github.com> Signed-off-by: dragongu <andrewgu@vip.qq.com> Signed-off-by: Andrew Grosser <dioptre@gmail.com> Signed-off-by: Justin Yu <justinvyu@anyscale.com> Signed-off-by: abrar <abrar@anyscale.com> Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com> Signed-off-by: Yicheng-Lu-llll <luyc58576@gmail.com> Signed-off-by: Potato <tanxinyu@apache.org> Signed-off-by: Zac Policzer <zac@anyscale.com> Signed-off-by: Cuong Nguyen <128072568+can-anyscale@users.noreply.github.com> Signed-off-by: Alexey Kudinkin <ak@anyscale.com> Signed-off-by: Seiji Eicher <seiji@anyscale.com> Signed-off-by: 杨睿 <595403043@qq.com> Signed-off-by: dayshah <dhyey2019@gmail.com> Signed-off-by: axreldable <aleksei.starikov.ax@gmail.com> Signed-off-by: kaihsun <kaihsun@anyscale.com> Signed-off-by: Kai-Hsun Chen <kaihsun@apache.org> Signed-off-by: Timothy Seah <tseah@anyscale.com> Signed-off-by: kevin <kevin@anyscale.com> Signed-off-by: joshlee <joshlee@anyscale.com> Signed-off-by: 400Ping <fourhundredping@gmail.com> Signed-off-by: Dhyey Shah <dhyey2019@gmail.com> Signed-off-by: harshit <harshit@anyscale.com> Signed-off-by: anmol <anmol@anyscale.com> Signed-off-by: Kit Lee <7000003+wingkitlee0@users.noreply.github.com> Signed-off-by: JasonLi1909 <jasli1909@gmail.com> Signed-off-by: Jason Li <57246540+JasonLi1909@users.noreply.github.com> Signed-off-by: MengqingCao <cmq0113@163.com> Signed-off-by: sampan <sampan@anyscale.com> Signed-off-by: zac <zac@anyscale.com> Signed-off-by: Elliot Barnwell <elliot.barnwell@anyscale.com> Signed-off-by: Mengjin Yan <mengjinyan3@gmail.com> Signed-off-by: myan <myan@anyscale.com> Signed-off-by: Seiji Eicher <58963096+eicherseiji@users.noreply.github.com> Signed-off-by: Linkun <github@lkchen.net> Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu> Signed-off-by: Markus <msobkowski@hercules.ai> Signed-off-by: Gagandeep Singh <gdp.1807@gmail.com> Signed-off-by: akyang-anyscale <alexyang@anyscale.com> Signed-off-by: Alan Guo <aguo@anyscale.com> Signed-off-by: haotian <haotian@ebay.com> Signed-off-by: Howie Tien <sword865@163.com> Signed-off-by: will.lin <will.lin@anyscale.com> Signed-off-by: Richard Liaw <rliaw@berkeley.edu> Signed-off-by: Ryan O'Leary <ryanaoleary@google.com> Signed-off-by: Andrew Sy Kim <andrewsy@google.com> Signed-off-by: Matvei Pashkovskii <mpashkov@amd.com> Signed-off-by: Kishanthan Thangarajah <kshanth2101@gmail.com> Signed-off-by: my-vegetable-has-exploded <wy1109468038@gmail.com> Signed-off-by: Neil Girdhar <mistersheik@gmail.com> Signed-off-by: Nikhil Ghosh <nikhil@anyscale.com> Signed-off-by: win5923 <ken89@kimo.com> Signed-off-by: Stephanie wang <smwang@cs.washington.edu> Signed-off-by: Stephanie Wang <smwang@cs.washington.edu> Signed-off-by: cong.qian <cong.qian@anyscale.com> Signed-off-by: Lonnie Liu <95255098+aslonnie@users.noreply.github.com> Signed-off-by: doyoung <doyoung@anyscale.com> Signed-off-by: Doyoung Kim <34902420+landscapepainter@users.noreply.github.com> Signed-off-by: simonsays1980 <simon.zehnder@gmail.com> Signed-off-by: Rui Qiao <161574667+ruisearch42@users.noreply.github.com> Signed-off-by: Sampan S Nayak <sampansnayak2@gmail.com> Signed-off-by: vincenthhan <vincenthhan@tencent.com> Signed-off-by: jeffreyjeffreywang <jeffjeffreywang@gmail.com> Signed-off-by: irabbani <irabbani@anyscale.com> Signed-off-by: Ibrahim Rabbani <israbbani@gmail.com> Signed-off-by: avigyabb <avigyabb@stanford.edu> Signed-off-by: avibasnet31 <avigyabb@anyscale.com> Signed-off-by: Tanner Wood <tanwood@amazon.com> Signed-off-by: avigyabb <98926738+avigyabb@users.noreply.github.com> Signed-off-by: Ricardo Decal <rdecal@anyscale.com> Signed-off-by: Matthew <mweston3@illinois.edu> Signed-off-by: tianyi-ge <tianyig@outlook.com> Signed-off-by: can <can@anyscale.com> Signed-off-by: Alexey Kudinkin <alexey.kudinkin@gmail.com> Signed-off-by: ahao-anyscale <ahao@anyscale.com> Signed-off-by: Lehui Liu <lehui@anyscale.com> Signed-off-by: Mark Rossetti <marosset@microsoft.com> Signed-off-by: hejialing.hjl <hejialing.hjl@bytedance.com> Signed-off-by: qiwenju <qiwenju@xiaomi.com> Signed-off-by: Q文举 <39372502+orangeQWJ@users.noreply.github.com> Signed-off-by: ljstrnadiii <ljstrnadiii@gmail.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Lonnie Liu <95255098+aslonnie@users.noreply.github.com> Co-authored-by: Rueian <rueian@anyscale.com> Co-authored-by: Dhyey Shah <dhyey2019@gmail.com> Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com> Co-authored-by: Jiajun Yao <jeromeyjj@gmail.com> Co-authored-by: Matthew Owen <mowen@anyscale.com> Co-authored-by: Sagar Sumit <sagarsumit09@gmail.com> Co-authored-by: Xinyuan <43737116+xinyuangui2@users.noreply.github.com> Co-authored-by: Justin Yu <justinvyu@anyscale.com> Co-authored-by: Kai-Hsun Chen <kaihsun@anyscale.com> Co-authored-by: matthewdeng <matt@anyscale.com> Co-authored-by: goutamvenkat-anyscale <goutam@anyscale.com> Co-authored-by: vickytsang <vtsang@amd.com> Co-authored-by: Rui Qiao <161574667+ruisearch42@users.noreply.github.com> Co-authored-by: Elliot Barnwell <elliot.barnwell@anyscale.com> Co-authored-by: Yevet <xyiwen@google.com> Co-authored-by: Mao Yancan <maoyancan@u.nus.edu> Co-authored-by: Mao Yancan <yancan.mao@bytedance.com> Co-authored-by: Cuong Nguyen <128072568+can-anyscale@users.noreply.github.com> Co-authored-by: Srinath Krishnamachari <68668616+srinathk10@users.noreply.github.com> Co-authored-by: iamjustinhsu <140442892+iamjustinhsu@users.noreply.github.com> Co-authored-by: Alexey Kudinkin <alexey.kudinkin@gmail.com> Co-authored-by: dragongu <38997200+dragongu@users.noreply.github.com> Co-authored-by: Andrew Grosser <dioptre@gmail.com> Co-authored-by: Abrar Sheikh <abrar@anyscale.com> Co-authored-by: kourosh hakhamaneshi <31483498+kouroshHakha@users.noreply.github.com> Co-authored-by: Yicheng-Lu-llll <51814063+Yicheng-Lu-llll@users.noreply.github.com> Co-authored-by: Potato <tanxinyu@apache.org> Co-authored-by: Zac Policzer <zac@anyscale.com> Co-authored-by: Alexey Kudinkin <ak@anyscale.com> Co-authored-by: Seiji Eicher <58963096+eicherseiji@users.noreply.github.com> Co-authored-by: 杨睿 <595403043@qq.com> Co-authored-by: Ibrahim Rabbani <israbbani@gmail.com> Co-authored-by: Aleksei Starikov <aleksei.starikov.ax@gmail.com> Co-authored-by: Stephanie Wang <swang@cs.berkeley.edu> Co-authored-by: Timothy Seah <timothy.seah777@yahoo.com> Co-authored-by: Timothy Seah <tseah@anyscale.com> Co-authored-by: Kevin H. Luu <kevin@anyscale.com> Co-authored-by: Qiaolin Yu <liin1211@outlook.com> Co-authored-by: Joshua Lee <73967497+Sparks0219@users.noreply.github.com> Co-authored-by: Ping <fourhundredping@gmail.com> Co-authored-by: harshit-anyscale <harshit@anyscale.com> Co-authored-by: Anmol Singh <kshasingh@gmail.com> Co-authored-by: anmol <anmol@anyscale.com> Co-authored-by: Kit Lee <7000003+wingkitlee0@users.noreply.github.com> Co-authored-by: Jason Li <57246540+JasonLi1909@users.noreply.github.com> Co-authored-by: matthewdeng <matthew.j.deng@gmail.com> Co-authored-by: Mengqing Cao <cmq0113@163.com> Co-authored-by: Sampan S Nayak <sampansnayak2@gmail.com> Co-authored-by: sampan <sampan@anyscale.com> Co-authored-by: Sven Mika <svenmika1977@gmail.com> Co-authored-by: Mengjin Yan <mengjinyan3@gmail.com> Co-authored-by: Nary Yeh <60069744+machichima@users.noreply.github.com> Co-authored-by: lkchen <github@lkchen.net> Co-authored-by: Balaji Veeramani <balaji@anyscale.com> Co-authored-by: Markus <44006014+minosvasilias@users.noreply.github.com> Co-authored-by: czgdp1807 <gdp.1807@gmail.com> Co-authored-by: akyang-anyscale <alexyang@anyscale.com> Co-authored-by: Alan Guo <aguo@anyscale.com> Co-authored-by: Howie Tien <haotian@ebay.com> Co-authored-by: Balaji Veeramani <bveeramani@berkeley.edu> Co-authored-by: simonsays1980 <simon.zehnder@gmail.com> Co-authored-by: William Lin <SolitaryThinker@users.noreply.github.com> Co-authored-by: Richard Liaw <rliaw@berkeley.edu> Co-authored-by: Ryan O'Leary <113500783+ryanaoleary@users.noreply.github.com> Co-authored-by: Andrew Sy Kim <andrewsy@google.com> Co-authored-by: Matvei Pashkovskii <matvei.pashkovskii@amd.com> Co-authored-by: Kourosh Hakhamaneshi <kourosh@anyscale.com> Co-authored-by: Kishanthan Thangarajah <kshanth2101@gmail.com> Co-authored-by: yi wang <48236141+my-vegetable-has-exploded@users.noreply.github.com> Co-authored-by: Neil Girdhar <mistersheik@gmail.com> Co-authored-by: Nikhil G <nrghosh@users.noreply.github.com> Co-authored-by: Jun-Hao Wan <ken89@kimo.com> Co-authored-by: Kai-Hsun Chen <kaihsun@apache.org> Co-authored-by: Stephanie Wang <smwang@cs.washington.edu> Co-authored-by: coqian <cong.qian@anyscale.com> Co-authored-by: angelinalg <122562471+angelinalg@users.noreply.github.com> Co-authored-by: Doyoung Kim <34902420+landscapepainter@users.noreply.github.com> Co-authored-by: vincenthhan <46981434+BestVIncent@users.noreply.github.com> Co-authored-by: vincenthhan <vincenthhan@tencent.com> Co-authored-by: Jeffrey Wang <jeffrey31415926@gmail.com> Co-authored-by: jeffreyjeffreywang <jeffjeffreywang@gmail.com> Co-authored-by: Ibrahim Rabbani <irabbani@anyscale.com> Co-authored-by: avigyabb <98926738+avigyabb@users.noreply.github.com> Co-authored-by: avibasnet31 <avigyabb@anyscale.com> Co-authored-by: tannerdwood <71387269+tannerdwood@users.noreply.github.com> Co-authored-by: Tanner Wood <tanwood@amazon.com> Co-authored-by: Ricardo Decal <crypdick@users.noreply.github.com> Co-authored-by: Kamil Kaczmarek <kaczmarek.poczta@gmail.com> Co-authored-by: Hassam Ullah Sheikh <hassamsheikh1@gmail.com> Co-authored-by: MatthewCWeston <61944935+MatthewCWeston@users.noreply.github.com> Co-authored-by: Artur Niederfahrenhorst <attaismyname@googlemail.com> Co-authored-by: Artur Niederfahrenhorst <artur@anyscale.com> Co-authored-by: Tianyi <tianyig@outlook.com> Co-authored-by: gangsf <gang@anyscale.com> Co-authored-by: Gang Zhao <gang@gang-JQ62HD2C37.local> Co-authored-by: ahao-anyscale <ahao@anyscale.com> Co-authored-by: Lehui Liu <lehui@anyscale.com> Co-authored-by: Mark Rossetti <marosset@microsoft.com> Co-authored-by: Jialing He <burglarralgrub@gmail.com> Co-authored-by: hejialing.hjl <hejialing.hjl@bytedance.com> Co-authored-by: Cuong Nguyen <can@anyscale.com> Co-authored-by: Q文举 <39372502+orangeQWJ@users.noreply.github.com> Co-authored-by: qiwenju <qiwenju@xiaomi.com> Co-authored-by: Len Strnad <ljstrnadiii@gmail.com> Co-authored-by: Goku Mohandas <gokumd@gmail.com> Signed-off-by: Marco Stephan <marco@magic.dev>

## Why are these changes needed? Hugging Face Datasets v4.0.0 removed dataset scripts and the related “dynamic modules” mechanism (See huggingface/datasets#7592), so `datasets.load.init_dynamic_modules` no longer exists. As `ray.data._internal.datasource.huggingface_datasource` currently imports/calls this function unconditionally, it raises: ``` AttributeError: module 'datasets.load' has no attribute 'init_dynamic_modules' ``` With this PR, Ray detects the legacy initializer and only calls it on Datasets ≤3.x. On Datasets ≥4.0.0 the block becomes a no-op. The behavior for older Datasets remains unchanged. ## Related issue number NA ## Checks - [x] I've signed off every commit(by using the -s flag, i.e., `git commit -s`) in this PR. - [x] I've run `scripts/format.sh` to lint the changes in this PR. - [ ] I've included any doc changes needed for https://docs.ray.io/en/master/. - [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in `doc/source/tune/api/` under the corresponding `.rst` file. - [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/ - Testing Strategy - [ ] Unit tests - [ ] Release tests - [ ] This PR is not tested :( --------- Signed-off-by: Masahiro Tanaka <mtanaka@anyscale.com> Signed-off-by: Masahiro Tanaka <81312776+tohtana@users.noreply.github.com> Signed-off-by: Lonnie Liu <lonnie@anyscale.com> Signed-off-by: Rueian <rueian@anyscale.com> Signed-off-by: Rueian <rueiancsie@gmail.com> Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com> Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com> Signed-off-by: Matthew Owen <mowen@anyscale.com> Signed-off-by: Sagar Sumit <sagarsumit09@gmail.com> Signed-off-by: xgui <xgui@anyscale.com> Signed-off-by: Xinyuan <43737116+xinyuangui2@users.noreply.github.com> Signed-off-by: Kai-Hsun Chen <kaihsun@anyscale.com> Signed-off-by: Matthew Deng <matt@anyscale.com> Signed-off-by: Goutam V <goutam@anyscale.com> Signed-off-by: root <vtsang@amd.com> Signed-off-by: Rui Qiao <ruisearch42@gmail.com> Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com> Signed-off-by: Yiwen Xiang <xyiwen@google.com> Signed-off-by: Mao Yancan <yancan.mao@bytedance.com> Signed-off-by: Mao Yancan <maoyancan@u.nus.edu> Signed-off-by: Cuong Nguyen <can@anyscale.com> Signed-off-by: iamjustinhsu <jhsu@anyscale.com> Signed-off-by: iamjustinhsu <140442892+iamjustinhsu@users.noreply.github.com> Signed-off-by: dragongu <andrewgu@vip.qq.com> Signed-off-by: Andrew Grosser <dioptre@gmail.com> Signed-off-by: Justin Yu <justinvyu@anyscale.com> Signed-off-by: abrar <abrar@anyscale.com> Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com> Signed-off-by: Yicheng-Lu-llll <luyc58576@gmail.com> Signed-off-by: Potato <tanxinyu@apache.org> Signed-off-by: Zac Policzer <zac@anyscale.com> Signed-off-by: Cuong Nguyen <128072568+can-anyscale@users.noreply.github.com> Signed-off-by: Alexey Kudinkin <ak@anyscale.com> Signed-off-by: Seiji Eicher <seiji@anyscale.com> Signed-off-by: 杨睿 <595403043@qq.com> Signed-off-by: dayshah <dhyey2019@gmail.com> Signed-off-by: axreldable <aleksei.starikov.ax@gmail.com> Signed-off-by: kaihsun <kaihsun@anyscale.com> Signed-off-by: Kai-Hsun Chen <kaihsun@apache.org> Signed-off-by: Timothy Seah <tseah@anyscale.com> Signed-off-by: kevin <kevin@anyscale.com> Signed-off-by: joshlee <joshlee@anyscale.com> Signed-off-by: 400Ping <fourhundredping@gmail.com> Signed-off-by: Dhyey Shah <dhyey2019@gmail.com> Signed-off-by: harshit <harshit@anyscale.com> Signed-off-by: anmol <anmol@anyscale.com> Signed-off-by: Kit Lee <7000003+wingkitlee0@users.noreply.github.com> Signed-off-by: JasonLi1909 <jasli1909@gmail.com> Signed-off-by: Jason Li <57246540+JasonLi1909@users.noreply.github.com> Signed-off-by: MengqingCao <cmq0113@163.com> Signed-off-by: sampan <sampan@anyscale.com> Signed-off-by: zac <zac@anyscale.com> Signed-off-by: Elliot Barnwell <elliot.barnwell@anyscale.com> Signed-off-by: Mengjin Yan <mengjinyan3@gmail.com> Signed-off-by: myan <myan@anyscale.com> Signed-off-by: Seiji Eicher <58963096+eicherseiji@users.noreply.github.com> Signed-off-by: Linkun <github@lkchen.net> Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu> Signed-off-by: Markus <msobkowski@hercules.ai> Signed-off-by: Gagandeep Singh <gdp.1807@gmail.com> Signed-off-by: akyang-anyscale <alexyang@anyscale.com> Signed-off-by: Alan Guo <aguo@anyscale.com> Signed-off-by: haotian <haotian@ebay.com> Signed-off-by: Howie Tien <sword865@163.com> Signed-off-by: will.lin <will.lin@anyscale.com> Signed-off-by: Richard Liaw <rliaw@berkeley.edu> Signed-off-by: Ryan O'Leary <ryanaoleary@google.com> Signed-off-by: Andrew Sy Kim <andrewsy@google.com> Signed-off-by: Matvei Pashkovskii <mpashkov@amd.com> Signed-off-by: Kishanthan Thangarajah <kshanth2101@gmail.com> Signed-off-by: my-vegetable-has-exploded <wy1109468038@gmail.com> Signed-off-by: Neil Girdhar <mistersheik@gmail.com> Signed-off-by: Nikhil Ghosh <nikhil@anyscale.com> Signed-off-by: win5923 <ken89@kimo.com> Signed-off-by: Stephanie wang <smwang@cs.washington.edu> Signed-off-by: Stephanie Wang <smwang@cs.washington.edu> Signed-off-by: cong.qian <cong.qian@anyscale.com> Signed-off-by: Lonnie Liu <95255098+aslonnie@users.noreply.github.com> Signed-off-by: doyoung <doyoung@anyscale.com> Signed-off-by: Doyoung Kim <34902420+landscapepainter@users.noreply.github.com> Signed-off-by: simonsays1980 <simon.zehnder@gmail.com> Signed-off-by: Rui Qiao <161574667+ruisearch42@users.noreply.github.com> Signed-off-by: Sampan S Nayak <sampansnayak2@gmail.com> Signed-off-by: vincenthhan <vincenthhan@tencent.com> Signed-off-by: jeffreyjeffreywang <jeffjeffreywang@gmail.com> Signed-off-by: irabbani <irabbani@anyscale.com> Signed-off-by: Ibrahim Rabbani <israbbani@gmail.com> Signed-off-by: avigyabb <avigyabb@stanford.edu> Signed-off-by: avibasnet31 <avigyabb@anyscale.com> Signed-off-by: Tanner Wood <tanwood@amazon.com> Signed-off-by: avigyabb <98926738+avigyabb@users.noreply.github.com> Signed-off-by: Ricardo Decal <rdecal@anyscale.com> Signed-off-by: Matthew <mweston3@illinois.edu> Signed-off-by: tianyi-ge <tianyig@outlook.com> Signed-off-by: can <can@anyscale.com> Signed-off-by: Alexey Kudinkin <alexey.kudinkin@gmail.com> Signed-off-by: ahao-anyscale <ahao@anyscale.com> Signed-off-by: Lehui Liu <lehui@anyscale.com> Signed-off-by: Mark Rossetti <marosset@microsoft.com> Signed-off-by: hejialing.hjl <hejialing.hjl@bytedance.com> Signed-off-by: qiwenju <qiwenju@xiaomi.com> Signed-off-by: Q文举 <39372502+orangeQWJ@users.noreply.github.com> Signed-off-by: ljstrnadiii <ljstrnadiii@gmail.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Lonnie Liu <95255098+aslonnie@users.noreply.github.com> Co-authored-by: Rueian <rueian@anyscale.com> Co-authored-by: Dhyey Shah <dhyey2019@gmail.com> Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com> Co-authored-by: Jiajun Yao <jeromeyjj@gmail.com> Co-authored-by: Matthew Owen <mowen@anyscale.com> Co-authored-by: Sagar Sumit <sagarsumit09@gmail.com> Co-authored-by: Xinyuan <43737116+xinyuangui2@users.noreply.github.com> Co-authored-by: Justin Yu <justinvyu@anyscale.com> Co-authored-by: Kai-Hsun Chen <kaihsun@anyscale.com> Co-authored-by: matthewdeng <matt@anyscale.com> Co-authored-by: goutamvenkat-anyscale <goutam@anyscale.com> Co-authored-by: vickytsang <vtsang@amd.com> Co-authored-by: Rui Qiao <161574667+ruisearch42@users.noreply.github.com> Co-authored-by: Elliot Barnwell <elliot.barnwell@anyscale.com> Co-authored-by: Yevet <xyiwen@google.com> Co-authored-by: Mao Yancan <maoyancan@u.nus.edu> Co-authored-by: Mao Yancan <yancan.mao@bytedance.com> Co-authored-by: Cuong Nguyen <128072568+can-anyscale@users.noreply.github.com> Co-authored-by: Srinath Krishnamachari <68668616+srinathk10@users.noreply.github.com> Co-authored-by: iamjustinhsu <140442892+iamjustinhsu@users.noreply.github.com> Co-authored-by: Alexey Kudinkin <alexey.kudinkin@gmail.com> Co-authored-by: dragongu <38997200+dragongu@users.noreply.github.com> Co-authored-by: Andrew Grosser <dioptre@gmail.com> Co-authored-by: Abrar Sheikh <abrar@anyscale.com> Co-authored-by: kourosh hakhamaneshi <31483498+kouroshHakha@users.noreply.github.com> Co-authored-by: Yicheng-Lu-llll <51814063+Yicheng-Lu-llll@users.noreply.github.com> Co-authored-by: Potato <tanxinyu@apache.org> Co-authored-by: Zac Policzer <zac@anyscale.com> Co-authored-by: Alexey Kudinkin <ak@anyscale.com> Co-authored-by: Seiji Eicher <58963096+eicherseiji@users.noreply.github.com> Co-authored-by: 杨睿 <595403043@qq.com> Co-authored-by: Ibrahim Rabbani <israbbani@gmail.com> Co-authored-by: Aleksei Starikov <aleksei.starikov.ax@gmail.com> Co-authored-by: Stephanie Wang <swang@cs.berkeley.edu> Co-authored-by: Timothy Seah <timothy.seah777@yahoo.com> Co-authored-by: Timothy Seah <tseah@anyscale.com> Co-authored-by: Kevin H. Luu <kevin@anyscale.com> Co-authored-by: Qiaolin Yu <liin1211@outlook.com> Co-authored-by: Joshua Lee <73967497+Sparks0219@users.noreply.github.com> Co-authored-by: Ping <fourhundredping@gmail.com> Co-authored-by: harshit-anyscale <harshit@anyscale.com> Co-authored-by: Anmol Singh <kshasingh@gmail.com> Co-authored-by: anmol <anmol@anyscale.com> Co-authored-by: Kit Lee <7000003+wingkitlee0@users.noreply.github.com> Co-authored-by: Jason Li <57246540+JasonLi1909@users.noreply.github.com> Co-authored-by: matthewdeng <matthew.j.deng@gmail.com> Co-authored-by: Mengqing Cao <cmq0113@163.com> Co-authored-by: Sampan S Nayak <sampansnayak2@gmail.com> Co-authored-by: sampan <sampan@anyscale.com> Co-authored-by: Sven Mika <svenmika1977@gmail.com> Co-authored-by: Mengjin Yan <mengjinyan3@gmail.com> Co-authored-by: Nary Yeh <60069744+machichima@users.noreply.github.com> Co-authored-by: lkchen <github@lkchen.net> Co-authored-by: Balaji Veeramani <balaji@anyscale.com> Co-authored-by: Markus <44006014+minosvasilias@users.noreply.github.com> Co-authored-by: czgdp1807 <gdp.1807@gmail.com> Co-authored-by: akyang-anyscale <alexyang@anyscale.com> Co-authored-by: Alan Guo <aguo@anyscale.com> Co-authored-by: Howie Tien <haotian@ebay.com> Co-authored-by: Balaji Veeramani <bveeramani@berkeley.edu> Co-authored-by: simonsays1980 <simon.zehnder@gmail.com> Co-authored-by: William Lin <SolitaryThinker@users.noreply.github.com> Co-authored-by: Richard Liaw <rliaw@berkeley.edu> Co-authored-by: Ryan O'Leary <113500783+ryanaoleary@users.noreply.github.com> Co-authored-by: Andrew Sy Kim <andrewsy@google.com> Co-authored-by: Matvei Pashkovskii <matvei.pashkovskii@amd.com> Co-authored-by: Kourosh Hakhamaneshi <kourosh@anyscale.com> Co-authored-by: Kishanthan Thangarajah <kshanth2101@gmail.com> Co-authored-by: yi wang <48236141+my-vegetable-has-exploded@users.noreply.github.com> Co-authored-by: Neil Girdhar <mistersheik@gmail.com> Co-authored-by: Nikhil G <nrghosh@users.noreply.github.com> Co-authored-by: Jun-Hao Wan <ken89@kimo.com> Co-authored-by: Kai-Hsun Chen <kaihsun@apache.org> Co-authored-by: Stephanie Wang <smwang@cs.washington.edu> Co-authored-by: coqian <cong.qian@anyscale.com> Co-authored-by: angelinalg <122562471+angelinalg@users.noreply.github.com> Co-authored-by: Doyoung Kim <34902420+landscapepainter@users.noreply.github.com> Co-authored-by: vincenthhan <46981434+BestVIncent@users.noreply.github.com> Co-authored-by: vincenthhan <vincenthhan@tencent.com> Co-authored-by: Jeffrey Wang <jeffrey31415926@gmail.com> Co-authored-by: jeffreyjeffreywang <jeffjeffreywang@gmail.com> Co-authored-by: Ibrahim Rabbani <irabbani@anyscale.com> Co-authored-by: avigyabb <98926738+avigyabb@users.noreply.github.com> Co-authored-by: avibasnet31 <avigyabb@anyscale.com> Co-authored-by: tannerdwood <71387269+tannerdwood@users.noreply.github.com> Co-authored-by: Tanner Wood <tanwood@amazon.com> Co-authored-by: Ricardo Decal <crypdick@users.noreply.github.com> Co-authored-by: Kamil Kaczmarek <kaczmarek.poczta@gmail.com> Co-authored-by: Hassam Ullah Sheikh <hassamsheikh1@gmail.com> Co-authored-by: MatthewCWeston <61944935+MatthewCWeston@users.noreply.github.com> Co-authored-by: Artur Niederfahrenhorst <attaismyname@googlemail.com> Co-authored-by: Artur Niederfahrenhorst <artur@anyscale.com> Co-authored-by: Tianyi <tianyig@outlook.com> Co-authored-by: gangsf <gang@anyscale.com> Co-authored-by: Gang Zhao <gang@gang-JQ62HD2C37.local> Co-authored-by: ahao-anyscale <ahao@anyscale.com> Co-authored-by: Lehui Liu <lehui@anyscale.com> Co-authored-by: Mark Rossetti <marosset@microsoft.com> Co-authored-by: Jialing He <burglarralgrub@gmail.com> Co-authored-by: hejialing.hjl <hejialing.hjl@bytedance.com> Co-authored-by: Cuong Nguyen <can@anyscale.com> Co-authored-by: Q文举 <39372502+orangeQWJ@users.noreply.github.com> Co-authored-by: qiwenju <qiwenju@xiaomi.com> Co-authored-by: Len Strnad <ljstrnadiii@gmail.com> Co-authored-by: Goku Mohandas <gokumd@gmail.com> Signed-off-by: Douglas Strodtman <douglas@anyscale.com>

…oject#55908) ## Why are these changes needed? Hugging Face Datasets v4.0.0 removed dataset scripts and the related “dynamic modules” mechanism (See huggingface/datasets#7592), so `datasets.load.init_dynamic_modules` no longer exists. As `ray.data._internal.datasource.huggingface_datasource` currently imports/calls this function unconditionally, it raises: ``` AttributeError: module 'datasets.load' has no attribute 'init_dynamic_modules' ``` With this PR, Ray detects the legacy initializer and only calls it on Datasets ≤3.x. On Datasets ≥4.0.0 the block becomes a no-op. The behavior for older Datasets remains unchanged. ## Related issue number NA ## Checks - [x] I've signed off every commit(by using the -s flag, i.e., `git commit -s`) in this PR. - [x] I've run `scripts/format.sh` to lint the changes in this PR. - [ ] I've included any doc changes needed for https://docs.ray.io/en/master/. - [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in `doc/source/tune/api/` under the corresponding `.rst` file. - [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/ - Testing Strategy - [ ] Unit tests - [ ] Release tests - [ ] This PR is not tested :( --------- Signed-off-by: Masahiro Tanaka <mtanaka@anyscale.com> Signed-off-by: Masahiro Tanaka <81312776+tohtana@users.noreply.github.com> Signed-off-by: Lonnie Liu <lonnie@anyscale.com> Signed-off-by: Rueian <rueian@anyscale.com> Signed-off-by: Rueian <rueiancsie@gmail.com> Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com> Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com> Signed-off-by: Matthew Owen <mowen@anyscale.com> Signed-off-by: Sagar Sumit <sagarsumit09@gmail.com> Signed-off-by: xgui <xgui@anyscale.com> Signed-off-by: Xinyuan <43737116+xinyuangui2@users.noreply.github.com> Signed-off-by: Kai-Hsun Chen <kaihsun@anyscale.com> Signed-off-by: Matthew Deng <matt@anyscale.com> Signed-off-by: Goutam V <goutam@anyscale.com> Signed-off-by: root <vtsang@amd.com> Signed-off-by: Rui Qiao <ruisearch42@gmail.com> Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com> Signed-off-by: Yiwen Xiang <xyiwen@google.com> Signed-off-by: Mao Yancan <yancan.mao@bytedance.com> Signed-off-by: Mao Yancan <maoyancan@u.nus.edu> Signed-off-by: Cuong Nguyen <can@anyscale.com> Signed-off-by: iamjustinhsu <jhsu@anyscale.com> Signed-off-by: iamjustinhsu <140442892+iamjustinhsu@users.noreply.github.com> Signed-off-by: dragongu <andrewgu@vip.qq.com> Signed-off-by: Andrew Grosser <dioptre@gmail.com> Signed-off-by: Justin Yu <justinvyu@anyscale.com> Signed-off-by: abrar <abrar@anyscale.com> Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com> Signed-off-by: Yicheng-Lu-llll <luyc58576@gmail.com> Signed-off-by: Potato <tanxinyu@apache.org> Signed-off-by: Zac Policzer <zac@anyscale.com> Signed-off-by: Cuong Nguyen <128072568+can-anyscale@users.noreply.github.com> Signed-off-by: Alexey Kudinkin <ak@anyscale.com> Signed-off-by: Seiji Eicher <seiji@anyscale.com> Signed-off-by: 杨睿 <595403043@qq.com> Signed-off-by: dayshah <dhyey2019@gmail.com> Signed-off-by: axreldable <aleksei.starikov.ax@gmail.com> Signed-off-by: kaihsun <kaihsun@anyscale.com> Signed-off-by: Kai-Hsun Chen <kaihsun@apache.org> Signed-off-by: Timothy Seah <tseah@anyscale.com> Signed-off-by: kevin <kevin@anyscale.com> Signed-off-by: joshlee <joshlee@anyscale.com> Signed-off-by: 400Ping <fourhundredping@gmail.com> Signed-off-by: Dhyey Shah <dhyey2019@gmail.com> Signed-off-by: harshit <harshit@anyscale.com> Signed-off-by: anmol <anmol@anyscale.com> Signed-off-by: Kit Lee <7000003+wingkitlee0@users.noreply.github.com> Signed-off-by: JasonLi1909 <jasli1909@gmail.com> Signed-off-by: Jason Li <57246540+JasonLi1909@users.noreply.github.com> Signed-off-by: MengqingCao <cmq0113@163.com> Signed-off-by: sampan <sampan@anyscale.com> Signed-off-by: zac <zac@anyscale.com> Signed-off-by: Elliot Barnwell <elliot.barnwell@anyscale.com> Signed-off-by: Mengjin Yan <mengjinyan3@gmail.com> Signed-off-by: myan <myan@anyscale.com> Signed-off-by: Seiji Eicher <58963096+eicherseiji@users.noreply.github.com> Signed-off-by: Linkun <github@lkchen.net> Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu> Signed-off-by: Markus <msobkowski@hercules.ai> Signed-off-by: Gagandeep Singh <gdp.1807@gmail.com> Signed-off-by: akyang-anyscale <alexyang@anyscale.com> Signed-off-by: Alan Guo <aguo@anyscale.com> Signed-off-by: haotian <haotian@ebay.com> Signed-off-by: Howie Tien <sword865@163.com> Signed-off-by: will.lin <will.lin@anyscale.com> Signed-off-by: Richard Liaw <rliaw@berkeley.edu> Signed-off-by: Ryan O'Leary <ryanaoleary@google.com> Signed-off-by: Andrew Sy Kim <andrewsy@google.com> Signed-off-by: Matvei Pashkovskii <mpashkov@amd.com> Signed-off-by: Kishanthan Thangarajah <kshanth2101@gmail.com> Signed-off-by: my-vegetable-has-exploded <wy1109468038@gmail.com> Signed-off-by: Neil Girdhar <mistersheik@gmail.com> Signed-off-by: Nikhil Ghosh <nikhil@anyscale.com> Signed-off-by: win5923 <ken89@kimo.com> Signed-off-by: Stephanie wang <smwang@cs.washington.edu> Signed-off-by: Stephanie Wang <smwang@cs.washington.edu> Signed-off-by: cong.qian <cong.qian@anyscale.com> Signed-off-by: Lonnie Liu <95255098+aslonnie@users.noreply.github.com> Signed-off-by: doyoung <doyoung@anyscale.com> Signed-off-by: Doyoung Kim <34902420+landscapepainter@users.noreply.github.com> Signed-off-by: simonsays1980 <simon.zehnder@gmail.com> Signed-off-by: Rui Qiao <161574667+ruisearch42@users.noreply.github.com> Signed-off-by: Sampan S Nayak <sampansnayak2@gmail.com> Signed-off-by: vincenthhan <vincenthhan@tencent.com> Signed-off-by: jeffreyjeffreywang <jeffjeffreywang@gmail.com> Signed-off-by: irabbani <irabbani@anyscale.com> Signed-off-by: Ibrahim Rabbani <israbbani@gmail.com> Signed-off-by: avigyabb <avigyabb@stanford.edu> Signed-off-by: avibasnet31 <avigyabb@anyscale.com> Signed-off-by: Tanner Wood <tanwood@amazon.com> Signed-off-by: avigyabb <98926738+avigyabb@users.noreply.github.com> Signed-off-by: Ricardo Decal <rdecal@anyscale.com> Signed-off-by: Matthew <mweston3@illinois.edu> Signed-off-by: tianyi-ge <tianyig@outlook.com> Signed-off-by: can <can@anyscale.com> Signed-off-by: Alexey Kudinkin <alexey.kudinkin@gmail.com> Signed-off-by: ahao-anyscale <ahao@anyscale.com> Signed-off-by: Lehui Liu <lehui@anyscale.com> Signed-off-by: Mark Rossetti <marosset@microsoft.com> Signed-off-by: hejialing.hjl <hejialing.hjl@bytedance.com> Signed-off-by: qiwenju <qiwenju@xiaomi.com> Signed-off-by: Q文举 <39372502+orangeQWJ@users.noreply.github.com> Signed-off-by: ljstrnadiii <ljstrnadiii@gmail.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Lonnie Liu <95255098+aslonnie@users.noreply.github.com> Co-authored-by: Rueian <rueian@anyscale.com> Co-authored-by: Dhyey Shah <dhyey2019@gmail.com> Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com> Co-authored-by: Jiajun Yao <jeromeyjj@gmail.com> Co-authored-by: Matthew Owen <mowen@anyscale.com> Co-authored-by: Sagar Sumit <sagarsumit09@gmail.com> Co-authored-by: Xinyuan <43737116+xinyuangui2@users.noreply.github.com> Co-authored-by: Justin Yu <justinvyu@anyscale.com> Co-authored-by: Kai-Hsun Chen <kaihsun@anyscale.com> Co-authored-by: matthewdeng <matt@anyscale.com> Co-authored-by: goutamvenkat-anyscale <goutam@anyscale.com> Co-authored-by: vickytsang <vtsang@amd.com> Co-authored-by: Rui Qiao <161574667+ruisearch42@users.noreply.github.com> Co-authored-by: Elliot Barnwell <elliot.barnwell@anyscale.com> Co-authored-by: Yevet <xyiwen@google.com> Co-authored-by: Mao Yancan <maoyancan@u.nus.edu> Co-authored-by: Mao Yancan <yancan.mao@bytedance.com> Co-authored-by: Cuong Nguyen <128072568+can-anyscale@users.noreply.github.com> Co-authored-by: Srinath Krishnamachari <68668616+srinathk10@users.noreply.github.com> Co-authored-by: iamjustinhsu <140442892+iamjustinhsu@users.noreply.github.com> Co-authored-by: Alexey Kudinkin <alexey.kudinkin@gmail.com> Co-authored-by: dragongu <38997200+dragongu@users.noreply.github.com> Co-authored-by: Andrew Grosser <dioptre@gmail.com> Co-authored-by: Abrar Sheikh <abrar@anyscale.com> Co-authored-by: kourosh hakhamaneshi <31483498+kouroshHakha@users.noreply.github.com> Co-authored-by: Yicheng-Lu-llll <51814063+Yicheng-Lu-llll@users.noreply.github.com> Co-authored-by: Potato <tanxinyu@apache.org> Co-authored-by: Zac Policzer <zac@anyscale.com> Co-authored-by: Alexey Kudinkin <ak@anyscale.com> Co-authored-by: Seiji Eicher <58963096+eicherseiji@users.noreply.github.com> Co-authored-by: 杨睿 <595403043@qq.com> Co-authored-by: Ibrahim Rabbani <israbbani@gmail.com> Co-authored-by: Aleksei Starikov <aleksei.starikov.ax@gmail.com> Co-authored-by: Stephanie Wang <swang@cs.berkeley.edu> Co-authored-by: Timothy Seah <timothy.seah777@yahoo.com> Co-authored-by: Timothy Seah <tseah@anyscale.com> Co-authored-by: Kevin H. Luu <kevin@anyscale.com> Co-authored-by: Qiaolin Yu <liin1211@outlook.com> Co-authored-by: Joshua Lee <73967497+Sparks0219@users.noreply.github.com> Co-authored-by: Ping <fourhundredping@gmail.com> Co-authored-by: harshit-anyscale <harshit@anyscale.com> Co-authored-by: Anmol Singh <kshasingh@gmail.com> Co-authored-by: anmol <anmol@anyscale.com> Co-authored-by: Kit Lee <7000003+wingkitlee0@users.noreply.github.com> Co-authored-by: Jason Li <57246540+JasonLi1909@users.noreply.github.com> Co-authored-by: matthewdeng <matthew.j.deng@gmail.com> Co-authored-by: Mengqing Cao <cmq0113@163.com> Co-authored-by: Sampan S Nayak <sampansnayak2@gmail.com> Co-authored-by: sampan <sampan@anyscale.com> Co-authored-by: Sven Mika <svenmika1977@gmail.com> Co-authored-by: Mengjin Yan <mengjinyan3@gmail.com> Co-authored-by: Nary Yeh <60069744+machichima@users.noreply.github.com> Co-authored-by: lkchen <github@lkchen.net> Co-authored-by: Balaji Veeramani <balaji@anyscale.com> Co-authored-by: Markus <44006014+minosvasilias@users.noreply.github.com> Co-authored-by: czgdp1807 <gdp.1807@gmail.com> Co-authored-by: akyang-anyscale <alexyang@anyscale.com> Co-authored-by: Alan Guo <aguo@anyscale.com> Co-authored-by: Howie Tien <haotian@ebay.com> Co-authored-by: Balaji Veeramani <bveeramani@berkeley.edu> Co-authored-by: simonsays1980 <simon.zehnder@gmail.com> Co-authored-by: William Lin <SolitaryThinker@users.noreply.github.com> Co-authored-by: Richard Liaw <rliaw@berkeley.edu> Co-authored-by: Ryan O'Leary <113500783+ryanaoleary@users.noreply.github.com> Co-authored-by: Andrew Sy Kim <andrewsy@google.com> Co-authored-by: Matvei Pashkovskii <matvei.pashkovskii@amd.com> Co-authored-by: Kourosh Hakhamaneshi <kourosh@anyscale.com> Co-authored-by: Kishanthan Thangarajah <kshanth2101@gmail.com> Co-authored-by: yi wang <48236141+my-vegetable-has-exploded@users.noreply.github.com> Co-authored-by: Neil Girdhar <mistersheik@gmail.com> Co-authored-by: Nikhil G <nrghosh@users.noreply.github.com> Co-authored-by: Jun-Hao Wan <ken89@kimo.com> Co-authored-by: Kai-Hsun Chen <kaihsun@apache.org> Co-authored-by: Stephanie Wang <smwang@cs.washington.edu> Co-authored-by: coqian <cong.qian@anyscale.com> Co-authored-by: angelinalg <122562471+angelinalg@users.noreply.github.com> Co-authored-by: Doyoung Kim <34902420+landscapepainter@users.noreply.github.com> Co-authored-by: vincenthhan <46981434+BestVIncent@users.noreply.github.com> Co-authored-by: vincenthhan <vincenthhan@tencent.com> Co-authored-by: Jeffrey Wang <jeffrey31415926@gmail.com> Co-authored-by: jeffreyjeffreywang <jeffjeffreywang@gmail.com> Co-authored-by: Ibrahim Rabbani <irabbani@anyscale.com> Co-authored-by: avigyabb <98926738+avigyabb@users.noreply.github.com> Co-authored-by: avibasnet31 <avigyabb@anyscale.com> Co-authored-by: tannerdwood <71387269+tannerdwood@users.noreply.github.com> Co-authored-by: Tanner Wood <tanwood@amazon.com> Co-authored-by: Ricardo Decal <crypdick@users.noreply.github.com> Co-authored-by: Kamil Kaczmarek <kaczmarek.poczta@gmail.com> Co-authored-by: Hassam Ullah Sheikh <hassamsheikh1@gmail.com> Co-authored-by: MatthewCWeston <61944935+MatthewCWeston@users.noreply.github.com> Co-authored-by: Artur Niederfahrenhorst <attaismyname@googlemail.com> Co-authored-by: Artur Niederfahrenhorst <artur@anyscale.com> Co-authored-by: Tianyi <tianyig@outlook.com> Co-authored-by: gangsf <gang@anyscale.com> Co-authored-by: Gang Zhao <gang@gang-JQ62HD2C37.local> Co-authored-by: ahao-anyscale <ahao@anyscale.com> Co-authored-by: Lehui Liu <lehui@anyscale.com> Co-authored-by: Mark Rossetti <marosset@microsoft.com> Co-authored-by: Jialing He <burglarralgrub@gmail.com> Co-authored-by: hejialing.hjl <hejialing.hjl@bytedance.com> Co-authored-by: Cuong Nguyen <can@anyscale.com> Co-authored-by: Q文举 <39372502+orangeQWJ@users.noreply.github.com> Co-authored-by: qiwenju <qiwenju@xiaomi.com> Co-authored-by: Len Strnad <ljstrnadiii@gmail.com> Co-authored-by: Goku Mohandas <gokumd@gmail.com>

remove scripts altogether

e1d7055

lhoestq and others added 9 commits June 5, 2025 13:49

fix tests

c13a5f3

remove dynamic_modules_path

86b1079

remove hf_modules_cache

2ca9279

remove unused test

02b0c93

fix auth in decoding

c7c3e07

fix tests

d475ec8

Merge branch 'main' into remove-scripts-altogether

96b4b80

Merge branch 'main' into remove-scripts-altogether

4b0927a

minor

adbb492

lhoestq marked this pull request as ready for review June 9, 2025 16:45

lhoestq merged commit 784607d into main Jun 9, 2025
9 of 15 checks passed

lhoestq deleted the remove-scripts-altogether branch June 9, 2025 16:45

qnzhou mentioned this pull request Jul 18, 2025

Python package not working Thingi10K/Thingi10K#2

Closed

This was referenced Jul 18, 2025

Auto-merge option for convert-to-parquet #7527

Closed

Add --merge-pull-request option for convert_to_parquet #7556

Closed

Add documentation for PGLearn AI4OPT/ML4OPF#34

Open

ryokan0123 mentioned this pull request Aug 17, 2025

datasets のバージョンを 4.0.0 未満に固定 ghmagazine/llm-book#58

Merged

ItsTania mentioned this pull request Aug 17, 2025

Remove dataset scripts for HF datasets. UKGovernmentBEIS/inspect_evals#469

Closed

tohtana mentioned this pull request Aug 25, 2025

[data] skip loading dynamic modules for HF datasets >= v4.0.0 ray-project/ray#55908

Merged

8 tasks

khannurien mentioned this pull request Aug 26, 2025

Fix benchmarks for Datasets 4.0.0 confident-ai/deepeval#1977

Merged

ArneBinder mentioned this pull request Oct 1, 2025

update datasets to <4.0.0 ArneBinder/pie-datasets#212

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Remove scripts altogether #7592

Remove scripts altogether #7592

Uh oh!

lhoestq commented Jun 4, 2025 •

edited

Loading

Uh oh!

HuggingFaceDocBuilderDev commented Jun 4, 2025

Uh oh!

Uh oh!

jp1924 commented Jul 14, 2025 •

edited

Loading

Uh oh!

lhoestq commented Jul 15, 2025

Uh oh!

jp1924 commented Jul 16, 2025

Uh oh!

autosquid commented Aug 4, 2025

Uh oh!

autosquid commented Aug 4, 2025 •

edited

Loading

Uh oh!

lhoestq commented Sep 4, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Remove scripts altogether #7592

Remove scripts altogether #7592

Uh oh!

Conversation

lhoestq commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HuggingFaceDocBuilderDev commented Jun 4, 2025

Uh oh!

Uh oh!

jp1924 commented Jul 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lhoestq commented Jul 15, 2025

Uh oh!

jp1924 commented Jul 16, 2025

Uh oh!

autosquid commented Aug 4, 2025

Uh oh!

autosquid commented Aug 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lhoestq commented Sep 4, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

lhoestq commented Jun 4, 2025 •

edited

Loading

jp1924 commented Jul 14, 2025 •

edited

Loading

autosquid commented Aug 4, 2025 •

edited

Loading