KEMBAR78
Autofix by evgeniy-r · Pull Request #228 · dotenv-linter/dotenv-linter · GitHub
Skip to content

Conversation

@evgeniy-r
Copy link
Contributor

@evgeniy-r evgeniy-r commented Jun 30, 2020

I have implemented automatic correction of the contents of files based on the results of the check.
When displaying, all warnings are marked as Fixed or Unfixed.
If all warnings are fixed, exit code is 0.

Just one kind of checks is fixable for now (LowercaseKey), but I have create the scaffold for implementing all kinds of checks.

The CLI key for the autofix mode is -f/--fix.

A few comments about the design:

The separate trait Fix was made for fixes (I tried to keep some symmetry with checks), but it is possible to expand the trait Check itself with this functions too.

There are two groups of fixes: some can be implemented by working only with the line that caused warning, for others you need to work with the entire line collection.

The Fix trait allows you to work with both groups of fixes due to default function implementations.

✔ Checklist:

  • This PR has been added to CHANGELOG.md (at the top of the list);
  • Tests for the changes have been added (for bug fixes / features);
  • Docs have been added / updated (for bug fixes / features).

@mgrachev mgrachev linked an issue Jul 1, 2020 that may be closed by this pull request
@mgrachev
Copy link
Member

mgrachev commented Jul 1, 2020

@evgeniy-r Thank you for your contribution! 🚀

This PR has a lot of changes. I guess I need more time to review this feature.

@mgrachev mgrachev requested a review from a team July 2, 2020 09:38
@mgrachev mgrachev added this to the v2.2.0 milestone Jul 2, 2020
@mgrachev
Copy link
Member

mgrachev commented Jul 6, 2020

I added the field with a checker name to Warning (it is possible to determine the checker by the text of the message, but I don't like this approach).

I'd like to minimize the changes in this PR, so I suppose to transfer this item to another PR.

@evgeniy-r
Copy link
Contributor Author

I'd like to minimize the changes in this PR, so I suppose to transfer this item to another PR.

I'm agree, I will do it in a couple of days.

@evgeniy-r
Copy link
Contributor Author

@DDtKey, thank you for feedback!

Will this API accurately make it easy to scale the application (add new fixes for checks)?

Important question. Yes, I experimented a bit and I'm sure that it will be possible to implement all existing checks based on this approach.

Is there enough information in the warnings to correct various types of warnings?

Sometimes (in several checks) we will work with the entire line entry collection and will use warnings only for indication (Fixed).
These checks (preliminarily): DuplicatedKey, ExtraBlankLine, UnorderedKey.

Perhaps the fixers should still be able to work without a preliminary check, what do you think?

I’ll think it over, but it seems to me that in terms of performance, checking first and then fixing is not a problem. On the other hand, we clearly link checks and fixes now.

The fact is that we are unlikely to be able to use the information, let's say about the sorting order, we’ll just have to fix the whole file.

For some checks, this is true, for others - not.

@DDtKey
Copy link
Member

DDtKey commented Jul 6, 2020

I apologize, I deleted the question before I received the answer, because I answered myself to some extent 😞

I will return it to leave a story:

Thanks for the contribution, autofix are a really important part of development 🚀

I would like to consider this PR a little more when I will have a bit more free time.

It seems important to me to discuss the moment, with a fix using the Warnings.
The ability to categorize as “fixed” and “not fixed” really looks great! 🔥

But some questions worry me:

  1. Will this API accurately make it easy to scale the application (add new fixes for checks)?
  2. Is there enough information in the warnings to correct various types of warnings?

Perhaps the fixers should still be able to work without a preliminary check, what do you think?

The fact is that we are unlikely to be able to use the information, let's say about the sorting order, we’ll just have to fix the whole file. And roughly speaking the Warning information gives not so much in such cases. It also possibly affects performance, although not critically.

At the same time, it is absolutely correct to display warnings that we cannot correct or are ambiguous (for example: IncorrectDelimiterChecker).

Maybe I'm completely wrong, but I want to raise this issue to look into the distance 🤔

@evgeniy-r Thanks for the detailed answer, I agree that the approach justifies itself.

@evgeniy-r
Copy link
Contributor Author

I have splitted this PR into two (#234).
I can't push the branch to this repo and rebase this PR to another branch, so there are 2 commits for now (one for each PR).

@mgrachev
Copy link
Member

@evgeniy-r The PR #234 has been merged. Please, update this PR.

@evgeniy-r
Copy link
Contributor Author

I have updated PR.

@evgeniy-r
Copy link
Contributor Author

@mgrachev, @DDtKey thank you very much for your feedback.
I will make changes.

@evgeniy-r
Copy link
Contributor Author

I changed the output format and there is no need in Option<bool> anymore (since the output format of each message no longer depends on the fixing option). I will simplify the code.

@evgeniy-r
Copy link
Contributor Author

Also there is some problem with tests on Windows platform - I will look in more detail.

@evgeniy-r
Copy link
Contributor Author

Also there is some problem with tests on Windows platform - I will look in more detail.

I have fixed tests, everything is OK.

@codecov-commenter
Copy link

codecov-commenter commented Jul 11, 2020

Codecov Report

Merging #228 into master will decrease coverage by 0.20%.
The diff coverage is 95.76%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #228      +/-   ##
==========================================
- Coverage   97.35%   97.14%   -0.21%     
==========================================
  Files          16       18       +2     
  Lines        1510     1679     +169     
==========================================
+ Hits         1470     1631     +161     
- Misses         40       48       +8     
Impacted Files Coverage Δ
src/common.rs 96.00% <50.00%> (-2.13%) ⬇️
src/main.rs 82.85% <86.66%> (+1.90%) ⬆️
src/checks.rs 100.00% <100.00%> (ø)
src/fixes.rs 100.00% <100.00%> (ø)
src/fixes/lowercase_key.rs 100.00% <100.00%> (ø)
src/fs_utils.rs 100.00% <100.00%> (ø)
src/lib.rs 94.91% <100.00%> (+0.27%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e37ef8f...93e05f9. Read the comment docs.

@evgeniy-r
Copy link
Contributor Author

I seem to have answered all the comments.
I can rebase on the master branch, if it is needed.

@DDtKey
Copy link
Member

DDtKey commented Jul 12, 2020

I think, we should use something like that:

        let file_name = String::from(".env");
        let tempdir = tempdir().expect("create tempdir");
        let path = tempdir.path().join(file_name);

This solves the problem of unique names and concurrency.
How this works in integration tests.

@evgeniy-r
Copy link
Contributor Author

I was able to reproduce this.
If the tests (with temp_dir + same file name) are in the same test module, we easily come across this on Windows,
and I have it reproduced on Linux including
Moreover, this does not happen with the cargo test - --test-threads 1 command, which confirms the above.

👍 It is an excellent finding!
I agree that we should use randomized file names.

@evgeniy-r
Copy link
Contributor Author

I added tempdir usage to tests. Also rebased to the fresh master.

Copy link
Member

@DDtKey DDtKey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this improvement (tempdir)!
I have only one comment left

@DDtKey DDtKey requested a review from mgrachev July 16, 2020 08:20
@DDtKey DDtKey removed the blocked Blocked by something label Jul 16, 2020
@mgrachev mgrachev requested a review from mstruebing July 16, 2020 08:43
DDtKey
DDtKey previously approved these changes Jul 16, 2020
evgeniy-r and others added 2 commits July 22, 2020 16:53
Co-authored-by: Grachev Mikhail <work@mgrachev.com>
Co-authored-by: Grachev Mikhail <work@mgrachev.com>
@evgeniy-r
Copy link
Contributor Author

In that case we should print an error instead of panic. Maybe we should use the Result enum instead of Option

OK, I will take a look.

Copy link
Member

@mgrachev mgrachev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@mgrachev mgrachev requested a review from DDtKey August 6, 2020 09:26
@mgrachev mgrachev removed a link to an issue Aug 6, 2020
@mgrachev mgrachev merged commit cf17699 into dotenv-linter:master Aug 6, 2020
@mgrachev
Copy link
Member

mgrachev commented Aug 6, 2020

@evgeniy-r Thanks a lot! 🙏 You've done a great job! 🔥

@evgeniy-r
Copy link
Contributor Author

You are welcome!

@mgrachev mgrachev mentioned this pull request Aug 11, 2020
21 tasks
@evgeniy-r evgeniy-r deleted the autofix_base branch September 26, 2020 08:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

4 participants