KEMBAR78
CREATE_PROJECT: Rely on user-wide vcpkg integration for msvc. by CrossVR · Pull Request #2568 · scummvm/scummvm · GitHub
Skip to content

Conversation

@CrossVR
Copy link

@CrossVR CrossVR commented Oct 26, 2020

This change simplifies vcpkg usage, but will also require all msvc
devs to use vcpkg moving forward.

By removing ExecutablePath, LibraryPath and IncludePath we
can rely on user-wide vcpkg integration to set these up correctly.
We can then also use the VcpkgCurrentInstalledDir macro to
set up the few custom paths we need like the SDL2 include path.

The consequence of removing those paths is that it's no longer
possible to simply download a prebuilt archive of dependencies
and set the SCUMMVM_LIBS environment variable for msvc.

User-wide vcpkg integration can be set up with vcpkg integrate install,
but I expect that people familiar with vcpkg will already have that setup.

Build instructions after this change

  1. Clone the vcpkg repository in a place separately from ScummVM if you don't have it yet:
https://github.com/microsoft/vcpkg.git
  1. Open powershell in the vcpkg folder and execute the following commands:
./bootstrap-vcpkg.bat
./vcpkg integrate install
  1. Now install the dependencies for ScummVM by executing the following commands:
./vcpkg install --triplet x86-windows curl faad2 fluidsynth freetype fribidi libflac libjpeg-turbo libmad libmpeg2 libogg libpng libtheora libvorbis sdl2 sdl2-net zlib discord-rpc
./vcpkg install --triplet x64-windows curl faad2 fluidsynth freetype fribidi libflac libjpeg-turbo libmad libmpeg2 libogg libpng libtheora libvorbis sdl2 sdl2-net zlib discord-rpc
  1. Go to the ScummVM folder, compile create_project and run create_msvc.bat

  2. Open scummvm.sln with your preferred version of Visual Studio and compile the solution.

@dreammaster
Copy link
Member

I'm somewhat concerned with this pull request, given the lack of information provided. What exactly are the benefits of having it like this versus the current installation of libraries. Is it easier to install the necessary libraries using it? Currently, it's been a straightforward matter of just dropping the zip file on the wiki and setting an environment variable. What benefit does vcpkg. Also, is there any reason to make it mandatory? Why not have it as a command line parameter for those that want to use vcpkg, just like we have the recently added flag for using canonical library names.

@CrossVR
Copy link
Author

CrossVR commented Oct 27, 2020

The benefit of vcpkg is moving towards a model of managing dependencies similar to package managers on linux (vcpkg is officially supported by Microsoft). You simply execute a few commands and vcpkg will take care of downloading the source code and compiling it for your platform and your version of Visual Studio. And since vcpkg is becoming more and more popular a developer may already have the dependencies installed.

The major downside of the zip file on the wiki is that it's only compiled for Visual Studio 2015 and will require updating every time a new version of Visual Studio is released. The other downside is that it requires you to keep these compiled dependencies around separately for ScummVM when you may already have them installed on your system through vcpkg for other projects.

The reason for making it mandatory is that it's more likely to work correctly since it adapts to your needs. It also ensures there's no more need to maintain this zip file on the wiki.

If you're not yet familiar with vcpkg I've added the updated build instructions to the PR description so you can see what the workflow would look like after this change.

@lotharsm lotharsm requested a review from SupSuper October 27, 2020 07:50
@dreammaster
Copy link
Member

Thanks. That'll be helpful; I'll try it out within the next few days

@CrossVR
Copy link
Author

CrossVR commented Nov 1, 2020

I'm done making my changes, let me know your thoughts after you've tried it out.

Before (and if) this PR is merged we'll probably need to prepare the buildbot for it. I'll also help out by updating the build instructions on the wiki.

@lotharsm
Copy link
Member

lotharsm commented Nov 1, 2020

Buildbot doesn't use MSVC or create_project at all, for the Win32 targets it relies on mingw-w64 and good ol' ./configure; make :-)

@dreammaster
Copy link
Member

I tried doing the process on my laptop. The vcpkg installation and libraries installation went fine, though I did have to add glew to the list of libraries that get installed. Now it's complaining about missing nasm. I tried adding the scummvm_lib bin folder to the path, but it still doesn't find it. It's gotten a bit late, so I'll try again tomorrow, but I'm curious if your changes have, or will need, anything special for nasm.

@CrossVR
Copy link
Author

CrossVR commented Nov 2, 2020

I think I have nasm in my path so I probably didn't notice. Will update the PR soon so it can find the vcpkg nasm.

@dreammaster
Copy link
Member

Ah, of course. If there's a nasm package that can be installed just by including it in the install lines, and be accessible, that'll make things easier. I'll forgo fiddling around with my paths tonight, and wait to see what you can come up with.

On a different topic, granted my laptop isn't bleeding edge anymore, but installing the packages still does quite a bit of time. This PR is definitely worthwhile by unchaining us from requiring prebuilt libraries for a specific version that is gradually getting out of date, but I still think it'd be better if create_project is able to be set up to see if vcpkg is installed, and if not fall back on how the projects are currently. That way, existing developers who want to simply keep using the libraries zip (and environment variable) can keep doing so, so long as they don't have vcpkg installed. That's assuming that there's a way in code to detect if vcpkg is installed.. presumably checking for some registry entry

@CrossVR
Copy link
Author

CrossVR commented Nov 3, 2020

Actually it seems I was mistaken, there is no nasm package available through vcpkg. It does have a cmake command to retrieve it: https://github.com/microsoft/vcpkg/blob/master/docs/maintainers/vcpkg_find_acquire_program.md

Not sure how to use that with MSBuild however. Alternatively we could file an upstream pull request to add nasm as a package.

On a different topic, granted my laptop isn't bleeding edge anymore, but installing the packages still does take quite a bit of time.

That's a consequence of building the dependencies from source code. It's going to take a bit of time given the large amount of dependencies.

That's assuming that there's a way in code to detect if vcpkg is installed.. presumably checking for some registry entry

I don't think there is a reliable way to do that, if we go that route I'd be more in favor of your earlier suggestion of providing a command-line argument to go back to the old system.

Alternatively there is also the vcpkg import command that allows you to import pre-built binaries. Which is an option if we want to keep offering pre-built binaries.

Copy link
Contributor

@SupSuper SupSuper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why it's necessary to remove SCUMMVM_LIBS support? If you want to use the vcpkg integration it just works, the SCUMMVM_LIBS paths don't cancel it out. The only issue I ran into was the custom SDL include we use, which you seem to have addressed separately.

I evaluated making vcpkg mandatory when updating the zip, but I don't think it's there yet. The installation process is cumbersome for new users and it doesn't support every ScummVM dependency, since we often rely on old libraries for old games. Maybe when they finish manifest support and build it into VS like nuget.

The zip process isn't optimal but it is familiar. Visual Studio 2015 is ABI-compatible with later versions so I don't think there's any danger of upgrades breaking it anytime soon. And vcpkg export makes it fairly easy to update.

@CrossVR
Copy link
Author

CrossVR commented Nov 4, 2020

The only issue I ran into was the custom SDL include we use, which you seem to have addressed separately.

That is actually not separately addressed, if we define the SCUMMVM_LIBS paths as we did before through <ExecutablePath>, <LibraryPath> and <IncludePath> it will cancel out vcpkg integration and the $(VcpkgCurrentInstalledDir) macro.

Visual Studio 2015 is ABI-compatible

Only for dynamically linked libraries, I'm not sure if the ScummVM dependency zip file contains any static libraries (other than import libraries), but those are not compatible. It will also require multiple VC++ runtimes to be installed if you link a VS2019 project with VS2015 dependencies unless those dependencies statically link to their respective runtimes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants