KEMBAR78
Recursive search for `.env` files by DDtKey · Pull Request #223 · dotenv-linter/dotenv-linter · GitHub
Skip to content

Conversation

@DDtKey
Copy link
Member

@DDtKey DDtKey commented Jun 26, 2020

I think it is useful that the linter can search for all env files in the specified directory, including the subdirectories.
This does not prevent specifying specific files and directories input, as well as excluding them.

What do you think?

✔ Checklist:

  • This PR has been added to CHANGELOG.md (at the top of the list);
  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been added / updated (for bug fixes / features).

@DDtKey DDtKey requested a review from a team June 26, 2020 15:54
@DDtKey
Copy link
Member Author

DDtKey commented Jun 26, 2020

I found #87 issue, it's already discussed, but in fact, I'm not sure what to compare the linter with libraries for load environment variables is correctly (they have different goals).

Most of the linters that I know by default work recursively.
We can add a flag to turn off recursion or turn it on, as an option, but it seems to me that this approach is more suitable for a linter.

I will be glad to discuss this and look at it from another perspective.

@mstruebing
Copy link
Member

Like in #87 my mind didn't change.
I think for linters to be most usable they should check recursively but if there are valid reasons against I don't know.
One thing we wouldn't can do as other linters mostly do is to only check version control system tracked files, because .env-files usually are not version tracked.

@mgrachev
Copy link
Member

@DDtKey Thank you!

It would be great if we first discussed this feature before its implementation 🙏
Because, I don't see any reasons to do recursive search for .env files.

I have o lot of projects with .env files (several files per project) and I haven't had situations where I need to use recursive search.

Other linters like rubocop can search recursively because projects written in Ruby can have subdirectories with .rb files.

To use .env files in our projects we should use dotenv libraries whick look for .env files in the current directory:

Or we should use Docker Compose and keep .env files somewhere else (if we want it).
I use both of ways every day and don't need to use recursive search.

@DDtKey
Copy link
Member Author

DDtKey commented Jun 29, 2020

@DDtKey Thank you!

It would be great if we first discussed this feature before its implementation pray
Because, I don't see any reasons to do recursive search for .env files.

I have o lot of projects with .env files (several files per project) and I haven't had situations where I need to use recursive search.

Other linters like rubocop can search recursively because projects written in Ruby can have subdirectories with .rb files.

To use .env files in our projects we should use dotenv libraries whick look for .env files in the current directory:

* Ruby: https://github.com/bkeepers/dotenv

* Rust: https://github.com/dotenv-rs/dotenv

* JS: https://github.com/motdotla/dotenv

* PHP: https://github.com/vlucas/phpdotenv

* Haskell: https://github.com/stackbuilders/dotenv-hs

* Python: https://github.com/theskumar/python-dotenv

Or we should use Docker Compose and keep .env files somewhere else (if we want it).
I use both of ways every day and don't need to use recursive search.

Yes, my fault :)

But as I wrote above, loading variables is not exactly the same as a linter.

In my memory, there were cases when folders were grouped and had other files nested. And they were directly loaded by applications according to completely different scenarios.

For example, take clippy, because we can write many modules, and it will check everything, regardless of usage. Most linter do so.

In the end, I just do not see a problem in the presence of this function, it allows you to do the same thing, but also complements the feature 🤔

@DDtKey
Copy link
Member Author

DDtKey commented Jun 29, 2020

If this seems like an excess, okay, I will close the PR.
Thanks for the review!

@mgrachev
Copy link
Member

Yes, my fault :)

But as I wrote above, loading variables is not exactly the same as a linter.

In my memory, there were cases when folders were grouped and had other files nested. And they were directly loaded by applications according to completely different scenarios.

Usually all .env files are kept in the root directory of projects or in another directory.
Right now we can check these files with the command:

$ dotenv-linter ./another-directory .

Do you know projects where .env files are stored differently?
It would be nice if you gave an example.

For example, take clippy, because we can write many modules, and it will check everything, regardless of usage. Most linter do so.

I think, clippy is a bad example, because it works with projects written in Rust. In Rust projects we can have such structure:

├── src
│   ├── checks
│   │   ├── duplicated_key.rs
│   │   ├── ending_blank_line.rs
│   │   ├── extra_blank_line.rs
│   │   ├── incorrect_delimiter.rs
│   │   ├── key_without_value.rs
│   │   ├── leading_character.rs
│   │   ├── lowercase_key.rs
│   │   ├── quote_character.rs
│   │   ├── space_character.rs
│   │   ├── trailing_whitespace.rs
│   │   └── unordered_key.rs
│   ├── checks.rs
│   ├── common.rs
│   ├── fs_utils.rs
│   ├── lib.rs
│   └── main.rs

But, I haven't seen any projects with similar structure with .env files:

├── example.env
├── global.env
├── subdir1
│   ├── common.env
│   └── subdir3
│       └── sample.env
└── subdir2
    └── test.env

In the end, I just do not see a problem in the presence of this function, it allows you to do the same thing, but also complements the feature 🤔

I see some problems:

  1. Recursive search is slower than search in only one directory;
  2. I know a lot of projects that store their dependecies in some directories (vendor, node_modules) and after recursive search we may get some warnings from dependecies or their dependecies.

@mgrachev
Copy link
Member

mgrachev commented Jun 29, 2020

If this seems like an excess, okay, I will close the PR.
Thanks for the review!

I think we can add this feature, but only as an additional option.

@mstruebing What do you think about it?

Or don't we need this feature?

@mgrachev mgrachev added the discussion Discussion of something label Jun 29, 2020
@DDtKey
Copy link
Member Author

DDtKey commented Jun 29, 2020

I really saw projects with the following structure :

environments
├── config.env
├── module1
│   └── module.env
└── module2
    └── module.env

But with a deep level of nesting, probably not, I have not seen.
I do not say that this is common, but it happens and maybe someone is useful.

@DDtKey
Copy link
Member Author

DDtKey commented Jun 29, 2020

  1. I know a lot of projects that store their dependecies in some directories (vendor, node_modules) and after recursive search we may get some warnings from dependecies or their dependecies.

Such cases can always be handled with exclude (it works well in the current PR), but yes, we can to turn off recursion by default.
If a function is needed at all :)

@wesleimp
Copy link
Contributor

In my experience, like @DDtKey, I have seen many projects with several .env files, mainly when monorepo (e.g. lerna) is used for frontend. I think this function is very useful and is nice to have this feature.

About the implementation, when recursion is used, will there be any default exclude list (node_modules, vendor, etc)?

@mstruebing
Copy link
Member

I think adding it as an optional feature is okay.
I currently work in a monorepo for example where every sub-package hat it's own .env-file like @DDtKey already mentioned, for that case it would be useful.
But mostly it's not an issue as most projects have a single .env file at root level.

So yeah, I agree: optional feature 👍

@DDtKey
Copy link
Member Author

DDtKey commented Jun 30, 2020

@mgrachev Before doing any further actions for an optional feature, I would like to know your opinion.
If this is still useful, I will definitely make a change ☺️

@mgrachev
Copy link
Member

I agree with @mstruebing. Please, add an additional flag to run recursive search.

@mgrachev mgrachev removed the discussion Discussion of something label Jul 1, 2020
@mgrachev mgrachev added this to the v2.1.0 milestone Jul 2, 2020
@mgrachev mgrachev mentioned this pull request Jul 2, 2020
5 tasks
@DDtKey
Copy link
Member Author

DDtKey commented Jul 3, 2020

I agree with @mstruebing. Please, add an additional flag to run recursive search.

Added a flag, updated tests and documentation.
Recursive search only works when a flag is specified.

Glad to discuss possible codebase improvements ☺️

Copy link
Member

@mgrachev mgrachev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DDtKey Great work! 🚀

I've left some comments. Please, look at them.

@DDtKey DDtKey requested a review from mgrachev July 3, 2020 22:04
@mgrachev mgrachev requested a review from a team July 4, 2020 08:49
@mgrachev
Copy link
Member

mgrachev commented Jul 7, 2020

@mstruebing Please, take a look at these changes 👀

Copy link
Member

@mstruebing mstruebing left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks very clean and understandable to me 👍

@DDtKey DDtKey merged commit 37baad9 into dotenv-linter:master Jul 8, 2020
@DDtKey DDtKey deleted the recursive-paths branch July 8, 2020 10:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

4 participants