KEMBAR78
[doc][hackathon] To add Adadelta Optimizer to the documentation by iramazanli · Pull Request #63255 · pytorch/pytorch · GitHub
Skip to content

[doc][hackathon] To add Adadelta Optimizer to the documentation #63255

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

[doc][hackathon] To add Adadelta Optimizer to the documentation #63255

wants to merge 1 commit into from

Conversation

@iramazanli
Copy link
Contributor

@iramazanli iramazanli commented Aug 13, 2021 •
edited by pytorch-probot bot
Loading

It has been discussed before that adding description of Optimization algorithms to PyTorch Core documentation may result in a nice Optimization research tutorial. In the following tracking issue we mentioned about all the necessary algorithms and links to the originally published paper #63236.

In this PR we are adding description of AdaDelta Algorithm to the documentation. For more details, we refer to the paper here https://arxiv.org/abs/1212.5701

AdaDeltaalg

cc @vincentqb @iramazanli

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Aug 13, 2021 •
edited
Loading

🔗 Helpful links

💊 CI failures summary and remediations

As of commit 05b6943 (more details on the Dr. CI page):

🕵️ 2 new failures recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

See GitHub Actions build linux-xenial-cuda11.3-py3.6-gcc7 / test (default, 1, 2, linux.8xlarge.nvidia.gpu) (1/2)

Step: "Test PyTorch" (full log | diagnosis details | 🔁 rerun)

2021-09-10T19:18:15.4048537Z AssertionError: Items in the second set but not the first: 
2021-09-10T19:18:15.4030340Z   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 766, in test_wrapper
2021-09-10T19:18:15.4031903Z     return test(*args, **kwargs)
2021-09-10T19:18:15.4033885Z   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 801, in dep_fn
2021-09-10T19:18:15.4035823Z     return fn(slf, *args, **kwargs)
2021-09-10T19:18:15.4037891Z   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 949, in only_fn
2021-09-10T19:18:15.4039537Z     return fn(self, *args, **kwargs)
2021-09-10T19:18:15.4040675Z   File "test_ops.py", line 154, in test_dtypes
2021-09-10T19:18:15.4042219Z     self.assertEqual(supported_dtypes, claimed_supported, msg=msg)
2021-09-10T19:18:15.4044857Z   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_utils.py", line 1639, in assertEqual
2021-09-10T19:18:15.4046897Z     super().assertEqual(x, y, msg=self._get_assert_msg(msg, debug_msg=debug_msg))
2021-09-10T19:18:15.4048537Z AssertionError: Items in the second set but not the first:
2021-09-10T19:18:15.4049755Z torch.int16
2021-09-10T19:18:15.4050641Z torch.int32
2021-09-10T19:18:15.4051507Z torch.int64
2021-09-10T19:18:15.4052402Z torch.bfloat16
2021-09-10T19:18:15.4053384Z torch.float16
2021-09-10T19:18:15.4054282Z torch.bool
2021-09-10T19:18:15.4055129Z torch.uint8
2021-09-10T19:18:15.4056016Z torch.float32
2021-09-10T19:18:15.4056902Z torch.int8
2021-09-10T19:18:15.4059474Z torch.float64 : Attempted to compare [set] types: Expected: set(); Actual: {torch.int16, torch.int32, torch.int64, torch.bfloat16, torch.float16, torch.bool, torch.uint8, torch.float32, torch.int8, torch.float64}.

See GitHub Actions build linux-xenial-cuda11.3-py3.6-gcc7 / test (default, 2, 2, linux.8xlarge.nvidia.gpu) (2/2)

Step: "Test PyTorch" (full log | diagnosis details | 🔁 rerun)

2021-09-10T19:49:18.4049148Z RuntimeError: CUDA error: an illegal memory access was encountered 
2021-09-10T19:49:18.4040478Z ----------------------------------------------------------------------
2021-09-10T19:49:18.4041148Z Traceback (most recent call last):
2021-09-10T19:49:18.4042217Z   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_utils.py", line 1117, in wrapper
2021-09-10T19:49:18.4043046Z     method(*args, **kwargs)
2021-09-10T19:49:18.4044135Z   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 378, in instantiated_test
2021-09-10T19:49:18.4044970Z     raise rte
2021-09-10T19:49:18.4046059Z   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 373, in instantiated_test
2021-09-10T19:49:18.4046959Z     result = test(self, **param_kwargs)
2021-09-10T19:49:18.4047658Z   File "test_reductions.py", line 1427, in test_argminmax_multiple
2021-09-10T19:49:18.4048376Z     x[mask] = torch.tensor(max_val + 1, dtype=dtype)
2021-09-10T19:49:18.4049148Z RuntimeError: CUDA error: an illegal memory access was encountered
2021-09-10T19:49:18.4050226Z CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
2021-09-10T19:49:18.4051234Z For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
2021-09-10T19:49:18.4051687Z 
2021-09-10T19:49:18.4052388Z ----------------------------------------------------------------------
2021-09-10T19:49:18.4052942Z Ran 39 tests in 6.894s
2021-09-10T19:49:18.4053225Z 
2021-09-10T19:49:18.4053653Z FAILED (errors=1, skipped=2)
2021-09-10T19:49:18.4053991Z 
2021-09-10T19:49:18.4054414Z Generating XML reports...
2021-09-10T19:49:18.4128588Z Generated XML report: test-reports/python-unittest/test_reductions/TEST-TestReductionsCUDA-20210910194911.xml

🚧 2 ongoing upstream failures:

These were probably caused by upstream breakages that are not fixed yet.

This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

@iramazanli iramazanli force-pushed the adadelta_algorithm_doc branch 4 times, most recently from 7499cbf to 0e157af Compare August 15, 2021 17:38
@iramazanli iramazanli changed the title To add Adadelta Optimizer to the documentation [doc][hackathon] To add Adadelta Optimizer to the documentation Aug 15, 2021
@iramazanli iramazanli requested a review from albanD August 15, 2021 18:14
@iramazanli iramazanli force-pushed the adadelta_algorithm_doc branch 2 times, most recently from 9fbbcd5 to 8cf072a Compare August 24, 2021 19:31
@codecov
Copy link

codecov bot commented Aug 24, 2021 •
edited
Loading

Codecov Report

Merging #63255 (798996d) into master (65e6194) will decrease coverage by 3.09%.
The diff coverage is n/a.

❗ Current head 798996d differs from pull request most recent head 23fbbb6. Consider uploading reports for the commit 23fbbb6 to get more accurate results

@@            Coverage Diff             @@
##           master   #63255      +/-   ##
==========================================
- Coverage   66.81%   63.72%   -3.10%     
==========================================
  Files         695      698       +3     
  Lines       90845    90881      +36     
==========================================
- Hits        60701    57912    -2789     
- Misses      30144    32969    +2825

@iramazanli iramazanli force-pushed the adadelta_algorithm_doc branch 6 times, most recently from 990ec96 to 23fbbb6 Compare August 27, 2021 21:08
albanD
albanD reviewed Sep 7, 2021
View reviewed changes
torch/optim/adadelta.py Outdated
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not name these like the other optimizers (like exp_avg) I feel like this would be easier to read?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds good ! and done

@iramazanli iramazanli force-pushed the adadelta_algorithm_doc branch 2 times, most recently from 800e4c5 to cdd0cf2 Compare September 9, 2021 19:21
albanD
albanD reviewed Sep 9, 2021
View reviewed changes
torch/optim/adadelta.py Outdated
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use v_0 like the other algorithms: https://pytorch.org/docs/master/generated/torch.optim.RMSprop.html#torch.optim.RMSprop ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done !

@iramazanli iramazanli force-pushed the adadelta_algorithm_doc branch 2 times, most recently from 3cd6c2f to 3ed039c Compare September 9, 2021 22:11
@facebook-github-bot
Copy link
Contributor

@iramazanli has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@iramazanli iramazanli force-pushed the adadelta_algorithm_doc branch from 3ed039c to 79cf319 Compare September 10, 2021 16:43
@facebook-github-bot
Copy link
Contributor

@iramazanli has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@iramazanli iramazanli force-pushed the adadelta_algorithm_doc branch from 79cf319 to 88dddf9 Compare September 10, 2021 17:42
@facebook-github-bot
Copy link
Contributor

@iramazanli has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@iramazanli iramazanli force-pushed the adadelta_algorithm_doc branch from 88dddf9 to 05b6943 Compare September 10, 2021 18:53
@facebook-github-bot
Copy link
Contributor

@iramazanli has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@iramazanli merged this pull request in dafa0a5.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@albanD albanD albanD approved these changes

Assignees

No one assigned

Labels

cla signed docs-hackathon hackathon Merged module: optimizer Related to torch.optim

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@iramazanli @facebook-github-bot @albanD