Setting up Unreal Gamesync (An Epic Saga)
Background
Epic's Unreal Gamesync is a tool for syncing engine and editor updates across a game development team. It's not a requirement for using the Unreal Engine, but if your game project has a lot of custom editor or especially engine code, Gamesync will remove a lot of the pain of building and distributing changes out to your team. Gamesync also removes the need for everyone on your team to run Visual Studio locally, as they will no longer need to compile game code locally. This is useful and money-saving if you have a lot of artists on your team.
Getting Gamesync setup can be a bit tricky - Epic's documentation doesn't explain or walk you through the entire process, so I decided to write up a guide based on my own experience, conversations with Epic, forum posts, and debugging the Gamesync source in Visual Studio.
Perforce setup
You're free to use whatever version control system you want with your Unreal project, but if you want to uses Gamesync you must be in Perforce, and in a version that supports streams. Beyond this, there seems to be a widely-held belief amongst game devs that you need to structure your Perforce depot the way Epic does - this is not the case. I wanted to set my team's Perforce depot up in a way that accomodated several different projects and a lot of non-game related content, all in a single depot. I started with Andrew Haining's post on Gamasutra. This was a useful primer, but I couldn't get my head around Haining's depot structure as he seems to suggest a Gamesync-compatible depot must have the following base structure
//UE4/Engine
//UE4/Game
which is what Epic uses. My preferred structure was going to be
//acmegamecompany/main/.../Engine
//acmegamecompany/main/.../Game
where main is our mainline stream, and where there would be other stream names later as we branched. Within this there would be space for a multi-disciplined development team - a real-life game project consists of more than just the "game" project in Unreal Editor, so we needed place for concept art, sound, graphics and other stuff that had to be in version sync with the Unreal project, but was not directly required in the game editor. We also want a lot of additional content that lives entirely outside of this one game project, including other games and things not related to games at all. A flexible mainline stream could theoretically look like
//acmegamecompany/main/thegame/Blender
//acmegamecompany/main/thegame/Concept-art
//acmegamecompany/main/thegame/EngineSource
//acmegamecompany/main/thegame/Foley
//acmegamecompany/main/thegame/Project
//acmegamecompany/main/thegame/Project/Build/UnrealGameSync.ini
//acmegamecompany/main/thegame/Graphics
The EngineSource directory contains Epic's Unreal Engine source code along with our custom engine modifications. The Project directory contains Unreal Editor files - .uproject and all other Unreal Editor files, Blueprints, etc. Note the 2nd last line - Gamesync relies on a config file UnrealGameSync.ini which it looks for in a predetermined place, and this is where it needs to go relative to everything else.
Once we have our mainline stream set up, we use a virtual stream to juggle our content around so our workspace looks like an "Epic" one. The stream's paths are
import * //acmegamecompany/main/thegame/EngineSource/*
share thegame/EngineSource/...
share thegame/Project/...
and we remap required directories with
thegame/EngineSource/... Engine/...
thegame/Project/... Game/...
This gives us a workspace with the following important landmarks that Gamesync expects
/Engine
/Game
/Game/Build/UnrealGameSync.ini
/Setup.bat
/GenerateProjectFiles.bat
The latter two come from the /EngineSource directory, and are specific to Engine source pulled from Github. So far we've determined that Gamesync will not run properly if these files are not present in the workspace root.
Hooking it all up
To build our engine and project, we sync our virtual stream down to a build server, and in a cmd window in the root of that workspace run
Setup.bat
followed by
Engine\Build\BatchFiles\RunUAT.bat BuildGraph `
-Script=Engine/Build/Graph/Examples/BuildEditorAndTools.xml `
-Target="Submit To Perforce for UGS" `
-set:EditorTarget=thegameEditor `
-Set:ArchiveStream=//acmegamecompany/Dev-Binaries `
-p4 `
-submit
EditorTarget here is the internal name of your Unreal project. We need Visual Studio 2017 installed on the build server for this, along with Unreal's myriad requirements (I'll list those in a future post). Notice that we specify //acmegamecompany/Dev-Binaries as the stream to sync to. This is a separate mainline stream that we store only Gamesync binaries in, and which Gamesync effectively owns, as it is the only client submitting to or syncing from it.
With the above, we build and push binaries to Perforce, but we cannot retrieve them yet. We we start the Gamesync client and point it our virtual stream, we'll see that the "Sync precompiled editor" option is greyed out. This is because Gamesync doesn't yet know where to retrieve binaries from. We need to add the following to UnrealGameSync.ini
[//acmegamecompany/main/Project/thegame.uproject]
ZippedBinariesPath=//acmegamecompany/Dev-Binaries/++acmegamecompany+gamesync-Editor.zip
This part took a while to figure out - it seems GameSync ignores virtual stream structure and traces back to the mainline stream base. It wants the path of the uproject it will be syncing, and it wants the depot path it will pull binaries from. If your Gamesync's "Sync precompiled editor" option is still greyed out, you can mouse over it to see the full path Gamesync is using - make sure this matches the actual path of your binary pack.