Refs: #252496

• macOS (zsh) @DonJayamanne
• linux (bash) @joshspicer
• windows (pwsh) @amunger

Complexity: 5

Create Issue

Background

The new experimental settings github.copilot.chat.agent.terminal.allowList and github.copilot.chat.agent.terminal.denyList have been added that lets users auto-approve some commands. These are the default values:

"github.copilot.chat.agent.terminal.allowList": {
  "echo": true,
  "cd": true,
  "ls": true,
  "cat": true,
  "pwd": true,
  "Write-Host": true,
  "Set-Location": true,
  "Get-ChildItem": true,
  "Get-Content": true,
  "Get-Location": true
},
"github.copilot.chat.agent.terminal.denyList": {
  "rm": true,
  "rmdir": true,
  "del": true,
  "kill": true,
  "curl": true,
  "wget": true,
  "eval": true,
  "chmod": true,
  "chown": true,
  "Remove-Item": true
},

allowList is a set of commands or regexes that will mark a command as allowed, the denyList can then override the allowed command and force it to require approval.

The following is planned but not yet implemented:

• A dropdown on the confirm button that will allow more easily auto-approving similar commands in the future
• An opt-in setting that will allow an LLM to determine whether a command is safe to auto-approve. Once implemented, this will essentially just be treated as the command line being trusted via allowList, denyList will still have the final say.

Security considerations

This is not intended to be a foolproof mechanism to prevent certain things from running. The set of "dangerous commands" is essentially infinite and we have workspace trust protecting us against the primary prompt injection attack vector. We trust both the user to ask a question that won't sneakily do something dangerous and the user's selected LLM to not act maliciously. It is however meant to catch the vast majority of dangerous command lines by default.

Testing

⚠ The chat.tools.autoApprove setting completely disables this mechanism, so make sure you don't have that set.
⚠ There is some shell specific logic for pwsh and zsh, everything else is treated as sh. So you might hit problems if you test with fish for example which I wouldn't recommend.

• Read each setting, are they easy to understand?
• Any suggestions for additions or changes to the default allowList and denyList values?
• Test default settings are working
• Test basic command approval
• Test both strings and regex entries for both allowList and denyList. Try some complex regex's or ones that match the entire line (^...$)
• Test sub-command detection. For example foo && bar gets split into foo and bar, both of which need to be in allowList to approve
• Test inline command detection. For example echo "$(abc)" gets split into echo "$(abc)" and abc, both of which need to be in the allowList to approve
• Test setting merging by adding user-level settings and overriding or adding them in the workspace's .vscode/settings.json
• Test denyList in various circumstances, it should always override allowList
• Test "yolo mode" by allowing something like "/.+/" and removing all defaults from denyList.
• Any feedback on the approach?
• Exploratory testing