IE11 is no longer supported
We do not support Internet Explorer 11 and below. Please use a different web browser.
Roberts Space Industries ®

Transmission

General

ID:

17991

Comments:

63

Date:

February 15th 2021

Alpha 3.12 Postmortem

Alpha 3.12 Postmortem

On December 17, 2020, we launched Alpha 3.12: Assault on Stanton, which introduced a number of new features and changes, including refinery decks, AI improvements, and gas clouds. The following is a postmortem from the senior developers themselves, detailing what was delivered and their thoughts on how it went. As a side-note, this postmortem specifically focuses on the Alpha 3.12 patch – we will release a separate postmortem Comm-Link focusing on the XenoThreat dynamic event separately.

ALPHA 3.12 POSTMORTEM

Vehicle Team


What went well

The Vehicle Pillar primarily supported a lot of other teams’ work for Star Citizen Alpha 3.12, particularly with capital ship combat ahead of the Arlington Bounty, CS5 UEE Navy Idris, and XenoThreat events. We worked not just on how the vehicles function and perform (being the largest ships deployed to the servers yet) but also helped deal with the increased lethality of torpedoes with smarter and more effective AI countermeasure usage.

Players also benefitted from this work with the ability to choose the type of countermeasure and how many are fired in a burst, adding greater tactical choice to the act of diverting incoming missiles and torpedoes. We also added further HUD elements to allow players to see how many of each type they have left along with the current burst size.

The last change to countermeasures was an expansion of the Alpha 3.11 changes. This makes each countermeasure type work against all missile seeker types but changes how effective they are depending on the type and number of incoming missiles. Decoys replaced flares as a time-limited point distraction while noise, formerly chaff, became an area-of-effect signal blocker (more countermeasures launched provides a higher chance of spoofing). We also changed the missiles themselves to provide minor variance in their tracking so a successful countermeasure would not divert all missiles equally. In addition to controlling the burst size manually, we added a panic function that empties 25% of available countermeasures in an attempt to overpower a surge of incoming missiles.

What didn’t go so well

We discovered a lot of issues with the missile setup and balance that caused odd behaviors. However, we decided to leave this until missiles are converted to IFCS 2.0 in Alpha 3.13 as it requires a comprehensive ground-up retune of all missile behaviors. In the future, we want to expand countermeasures by giving players better feedback on their own signatures and missile abilities, which will start coming online with Missile Operator Mode.

Vehicle Entry Identification was a much-requested quality-of-life feature (building upon the ASOP Hangar Spawn notifications of Alpha 3.11) that allows players to quickly identify the points of entrance into a ship. These markers update depending on whether the ship is landed or in zero-g, removing a common complaint new players had of being unable to figure out how to get into their ship. Occasionally these displays wildly offset from the vehicle, which we’re looking into, but that was pretty much the only negative issue with the feature and it was generally well-received.

The main content delivery in Alpha 3.12 was the Esperia Talon and Talon Shrike, which are a pair of light fighters that expand the line-up in-game. Generally, these went pretty well and we discussed their development in multiple ISC episodes, including our work on the Hard Surface Shader and iridescent materials.

Unfortunately, a few issues were present at release that we’re aiming to fix in future patches. These include the screen requiring toggling in Arena Commander (also present on the Prowler) and the Talon Shrike’s missile launchers sometimes stopping functioning after a large number have been fired.

John Crewe
Vehicle Director

USPU Gameplay Feature Team

Q4 for the USPU team is always a busy one. Not just for us, but for the entire company. Here’s a brief summary of our more important initiatives that we were able to get into your hands during the quarter. Thankfully, for better or for worse, we didn’t have a massive CitizenCon demo to prepare for this year, but that didn’t stop us from being extremely busy.

International Aerospace Exposition (IAE)

What went well

We successfully extended our IAE content into the high-tech art style, which took place at New Babbage on planet microTech. We added several new things to the event this year that tried to cater to comments from the fans. Firstly, we had each manufacturer’s expo hall run for two consecutive days in case fans missed the first and extended the free rental period from one day to two. Hopefully these two things allowed everyone the opportunity to rent the ships they were eager to try. We also had two expo halls running on any given day. This obviously allowed for more things for the players to see and, in the spirit of “more things to see,” we decided to showcase many of our ship components and weapons in the main hall. Between two halls active each day and all the additional items, this was a true testament that our tech is getting more and more optimized as we would never have been able to do that in the past. Finally, to try and extend accessibility, we added a series of rental kiosks into the “Best In Show” hall, which ran for four days. In these kiosks, we put every vehicle that was shown throughout the course of the show and gave the same two-day rental period. Between all of this, we feel it was the most engaging iteration we’ve created to date.

What didn’t go so well

We had two vehicles that were introduced to the game at the IAE show that really came down to the wire in terms of their development. Ultimately, this didn’t allow us to lock down the build until a few days before the show. Because this show is open to the public in November, it had to be released as a point-patch to the Alpha 3.11 build. Not being able to lock down the build before we branched to the Alpha 3.12 build caused some file-management headaches. Critical fixes are still being made to the point release and those same fixes also need to be in the newly branched stream content. This inevitably leads to work getting stomped here and there and, at the end of the day, eats valuable time that could be used to fix other bugs or make new things. Also, some things that we intended to keep secret until a bit closer to the time of the event leaked. This was hardly a major issue, but it’s always nice to be able to surprise the fans from time to time.

What we’ll do differently in the future

A few things I really want to focus on as we move forward with this event are:

  • Modularity/re-use of assets from previous events. The faster we can make these events happen, the more time we’ll have to focus on new content ideas or other solar systems.
  • Preproduction planning. We know the event will happen again next November, so I would like to iron out things like the color scheme, event logos, location, etc. as early as possible. This way, when it comes time to work on the content, nobody is waiting for anything to be decided and we can just put our heads down and work.
  • Get all content for the event into the build so we can avoid requiring a point release. As mentioned above, having two release streams/branches running in parallel is simply asking for trouble.

Reputation System

What went well

Another important system that was added in Q4 was the reputation system that we’ve always intended to have. While this isn’t visible to players yet, they are able to experience it through our mission givers and some of our mission chains (the bounty hunter chain is the most notable series that is currently being unlocked by reputation gains). Not every mission has been converted to the new system, though it will be an ongoing process over the next quarter or two. This new reputation system will be the foundation for a significant number of gameplay systems as we move forward. Not only will this be the main way that content is released to players, it will also be tied to things like NPC responses (currently seen in mission givers), perks and benefits that can be earned by participating in content, for tracking player progress through career paths, for dictating relationships between the organizations within the game, and a number of other essential gameplay loops. We felt that this was so important to get into the game that we elected to release it without any player facing UI (which is in progress for Q1/Q2 release). But because this is going to be so ingrained in a significant number of systems moving forward, we felt it was worth the sacrifice. Not only will it be represented in a standalone reputation UI that will allow players to view their career tracks with various organizations, it will also track the key NPCs they have interacted with along with their current standing with each. Reputations will be exposed in this UI as they come across them in the universe so that it encourages players to travel around and engage with the content to expose them. Additionally, we will be revamping the Mission Manager to include player visibility as to what sort of reputation requirements they need to acquire missions. Reputation is going to be one of the largest progression mechanics that Star Citizen will have going for it outside of the economy since we are a skill-based sandbox game not driven by “leveling” or “talent tree” systems. Reputation is now service-driven on the backend as well. This means that all reputation gains can be preserved between patches, which will be a great thing for the players.

What didn’t go so well

As far as general development of this feature, it went quite smoothly once we got to dig our heels into it. We had started work on it about a year prior to this but unfortunately got pulled onto some higher priority stuff at the time. If I had it to do over, I would have kept the team on this until it was done and had the desired UI/UX changes to go along with it. Not having the UI in place with its release, while not game-breaking, is a critical piece of this feature. Ensuring that all our features are intuitive often relies on this type of visual feedback.

What we’ll do differently in the future

Moving forward, I would like to try and budget the required time to release a more “complete” feature. While it’s not always possible to convert all of a game’s existing content over to new systems like this, I would like to try and make sure we can do more when big changes like these happen in the future. I was happy that we got more than the original intention into place but would have loved to get more time with the Mission Team to help convert even more than we did. The next time something as foundational as this comes up, I would like to personally do a better job of raising global awareness across the company so we can convert as much of it as possible.

Front End Conversion to Building Blocks

What went well

We were able to convert the initial two screens in the frontend to utilize our Building Blocks tech. In general, I think this process went fairly well. It didn’t require a ton of people and was a fairly self-contained bit of work. We also got the opportunity to fix a few things that we wanted to address since adding the “new friends” service back in Q1 2020. Switching this over to our new UI system also allowed us to fade all of the UI elements together. Going through this process gave us some time to critically think through the frontend since the project has evolved so much over the last few years. We also set it up so that the current solution can scale up as we add more solar systems. We cleaned up the main screen so that we can have a greater view of the background images. I’m happy that we were able to attribute a different image for each possible landing zone; I think it really helps give new players a sense of what type of location they’re choosing.

What didn’t go so well

The frontend is very ingrained with the original CryEngine toolset that we originally started with. The Building Blocks system works based on entities, which means that our core data needs to be loaded so that we have an entity to run the canvas on. On top of that, the original system requires a level to be loaded. Because of this, we were not able to replace any elements prior to selecting what game mode players want to play (at least not with Building-Blocks-based tech/implementation). The ultimate solution is to completely rework this code base but that was way outside the scope of what we were able to deal with, nor was it our main directive here.

What we’ll do differently in the future

The bottom line here is that the game that we based our original frontend on is much different from the game that we are building towards today. I’m not sure how much the USPU Team will be changing the frontend in the future, but the next time we make changes here I’d really like to be working towards the final vision. This would allow us to really streamline the new user experience so that getting into the game for the first time, or back into the game for returning players, is as simple and intuitive as possible. Because we have been bolting on new features one at a time, this part of the game has suffered as a result. If this can be redefined from the ground up, there’s a lot that we’d do differently based on how much the game has evolved.

Rob Reininger
Lead System Designer and USPU Product Owner

Systemic Services & Tools Team

What went well

The Systemic Services and Tools (SST) Team continued working on the Quantum simulation and integrating it into services alongside internal presentations of new technology that we’re excited to share with the community soon.

Administrative work occurred last quarter to shift around internal CIG resources for SST. The team will be increasing in size to accommodate the growing need for services and support for Quantum across many aspects of the game.

Outside of services and simulation work, SST created new tools to support the upcoming reputation system and the way reputation is distributed across the game universe. SST continues to support the Star Citizen economy with Data Tools to help alleviate the massive amount of data while we prepare for Quantum to take the reins.

What didn’t go so well

As we transition into a larger department, we had some growing pains with our response times to other teams. Features like the refinery service didn’t get the immediate attention they needed while our attention was elsewhere.

What we’ll do differently in the future

We’re looking for ways to better streamline and distribute the work coming into SST for the growing team. In addition, we’ve set up automated messaging to supplement the communication coming out of SST to dependent teams.

Jake Muehle
Senior Technical Designer

Design Team

AI Intercepting Torpedoes

What went well

Turrets on the Idris intercepting torpedoes works well, and it creates some very cool moments when they successfully intercept.

What didn’t go so well

AI accuracy is not good enough to reliably shoot down incoming torpedoes depending on server framerate.

What we’ll do differently in the future

We will look to improve the AI accuracy so we can have greater control over how many torpedoes slip through turret defenses.

AI Fire Mode Usage and Targeting Priorities

What went well

The AI using burst-fire greatly improves the look of AI turret fire. Plus, the targeting priorities ensure that the AI are attacking a sensible target for their ship class/turret size.

What didn’t go so well

At the minute, using burst-fire puts the AI at a disadvantage compared to the players as fully automatic firing increases damage.

What we’ll do differently in the future

We will rebalance AI burst-firing when capacitors are introduced to reduce the effectiveness of holding down the fire button.

AI Accuracy Convergence

What went well

The new accuracy system acts in a more believable manner as it tracks the target’s position and continues to fire at that position until the target moves. This is preferable to the old system where the AI’s aim would swing wildly as it attempted to miss stationary targets in front of it.

What didn’t go so well

The AI is not accurate enough to pose a decent level of threat to the player at the minute. And, the new system tends to miss behind the target’s movement instead of in front, so the players don’t see that they are being shot at as much.

What we’ll do differently in the future

We want to improve the accuracy overall and make it so different NPCs have a more noticeable accuracy difference between skilled and unskilled AI. The accuracy system will also be iterated upon to make it overshoot as opposed to undershoot the target more often.

Capital Ship Combat Behavior

What went well

The capital ships now put a good amount of consideration into distance and relative position when engaging other ships. Capital ships without frontal guns will attempt to get in close enough to utilize all of their turrets, while those with large frontal guns will attempt to keep out of the enemy’s range and utilize their powerful long-ranged weapons.

What didn’t go so well

The tactic selection doesn’t consider the AI ship’s strength relative to the target enough. For example, when the Idris is fighting a gunship or capital ship, it will attempt to stay at a distance and use its railgun. This often makes sense but can lead to it running away from smaller gunships that it could easily take in a close-range slap fight.

What we’ll do differently in the future

We will iterate on the tactic selection so the AI consider their own ship and target in more detail than just the ship class. Also, we will want to allow pilot character traits to affect the capital ship behavior as well.

Elevator Panels

What went well

We successfully laid the groundwork for future elevator panels (and other re-styleable screens), making them inherit their styles from the transit system and be usable on any shaped canvas. This means all future panels can use the same two files but still look different.

What didn’t go so well

The transit system is very difficult to set up and test in its current form – the UI Team were unable to get it working properly and had to rely on Level Design to implement. However, UI and Level Design didn’t work on the feature at the same time, resulting in work starting and stopping in different streams and bugs getting missed. New Building Blocks styles are extremely time-consuming to implement due to a lack of tools and an inability to merge files.

What we’ll do differently in the future

We’ll make sure that the feature is developed, implemented, and tested in one go in a feature stream so that bugs are picked up and fixed before it’s “finished.” We’ll also make sure the team that owns the feature has time to fix code issues as part of the feature development cycle and have the UI Team focus on the UI.

Station Based Refining

What went well

We finished the initial gameplay loop for refining, complete with refineries having their own material specializations, workloads, and the ability to refine, collect, and sell refined materials at various places across the galaxy. The refinery decks themselves look spectacular and the UI for the refinery terminal itself is in a great place to expand upon the gameplay loop with very little in the way of reworking things, which should mean quicker iteration down the line.

What didn’t go so well

Our planning for every step was extremely thorough, however, due to several situations out of our control, we were unable to reach a point early enough where we could balance the refinery loop the way we intended to. So, we brought forward another already-intended set of changes in the form of the initial mining rebalance to compensate as best we could until we can get the refinery balance to the level we originally wanted. This mining rebalance had the added bonus that it made the MISC Prospector a bit more appealing for people to start out with or rent as well as providing more gameplay experiences for the Argo MOLE or multiple players with Prospectors.

What we’ll do differently in the future

Get the UI art play-tested earlier. Some players didn’t know what parts of the screen they could interact with and this would have allowed us to have more time to make changes.

Mining UI Refactor

What went well

We reworked the entire mining UI to work with Building Blocks. This went a lot smoother than any of us could have ever hoped as a lot of the mining gameplay loop setup actually worked with Building Blocks straight out of the box without much reworking at all. This gave us leeway to add more to the mining UI than we originally intended to, meaning we were able to not only provide an entirely new UI that’s scalable across three different mining vehicles, but we showed off that scalability by quickly iterating on UI canvas pieces to support previously introduced systems. This included volatile cargo and added an entirely new cargo hold piece that further provides players with information we always wanted to provide but never had the ability to do so.


What didn’t go so well

The initial pass of the mining UI refactor was trickier to implement than it was to build, with different vehicles having different spaces available to them for the UI, meaning that a lot of behind-the-scenes tweaking was required to actually display the UI in a comfortable way. This is still being fine-tuned and, with future tech coming soon, we hope to broadcast the UI in a slightly more visually appealing way that works better than the current implementation.

What we’ll do differently in the future

We’ll iterate more quickly in the future on things like this as we now have a firmer understanding of Building Blocks and its benefits.

Todd Papy
Star Citizen Live Director

Core Gameplay Pillar

Multi-Tool Tractor Beam

The Multi-Tool Tractor Beam is an exciting addition to the ‘verse and is a core piece of functionality that unlocks several gameplay loops that we are working on, such as cargo hauling and EVA traversal spaces. The main use-case for the Tractor Beam in Alpha 3.12 is to allow for easier collection of cargo boxes in EVA either during lost in space missions or for the collection of post-ship-combat loot. While on the surface the Tractor Beam is a point-and-shoot tool, under the hood a lot is going on, and I think the team did a fantastic job of creating a truly systemic feature that is accessible and easy to use.

The first challenge that we faced is that we wanted it to work in several different gravity settings and to utilize the weight of an object. This led to several issues, as in zero-g everything is weightless, so it meant that you could potentially move something huge that weighed very little. For example, a planet-sized polystyrene ball. We designed the Tractor Beam around two core concepts – volume and force. Volume dictates the general size of the object you can lift while force dictates the amount of effort the gun needs to apply to lift the object. This means that for any item that has a mass and physics collider (which every item should already have), the force required to lift it is automatically calculated using the environment’s gravity. This allowed us to build a feature that works with any physics object in the game without having to do lots of manual set up.

The second challenge was how do we explain to the player what they can and cannot lift without having to ADS on everything or even worse, use trial and error. Fortunately, the Multi-Tool already has a small screen on the back that allowed us to implement some simple iconography alongside a traffic light color-coding system. This means that we’re able to clearly show all five states of an object by just looking at it:
  1. Object can be lifted
  2. Object can be lifted but is out of range
  3. Object is too heavy
  4. Object can never be lifted
  5. You will travel to the object

While we did provide additional information in the ADS view, everything that you need to know is on the back of the tool and I was really pleased we were able to distill so much information into such a small screen.


What didn’t go so well

When planning any feature, you want to identify the core experience and then outline any secondary mechanics that would enhance that experience. For example, a jump mechanic by itself is a standalone feature, but you may decide that in-air control (a secondary mechanic) enhances that core experience. With the Tractor Beam, we sat down and identified the core experience of being able to manipulate objects and use it as a grapple hook in EVA, and I feel we delivered on this. But there were some secondary mechanics that I would have liked to ship that we ran out of time to deliver. Specifically, allowing the manipulation of your trajectory using your suit thrusters when using the grapple hook functionality in zero-G.

Unfortunately, the two systems did not really work well together, with the force used to pull you along overriding any force used by the suit thrusters. It also didn’t help that EVA was also being moved over to using IFCS at the same time and this led to us having to make a priority call to focus on the core experience and make sure that was as polished as can be for release. This happens all the time with features and is the nature of game development, but it was a shame it missed the first iteration. We will add this functionality in a later release.

What we’ll do differently in the future

Delivering a feature for release requires lots of different teams coordinating together, from VFX to Audio to the feature team working on the functionality. One of the biggest things I am trying to improve upon is to give the mission designers more time to allow them to integrate all of these different features into the ‘verse in a meaningful way. If you have read my previous postmortems, you will probably see a trend that this is something I am actively trying to improve, but as multiple teams and schedules are involved, it takes a little bit of time to readjust. If I could go back in time, I would probably have tried to get a simple version into the hands of the mission/level designers sooner so they could have played with it a little bit more.

Weapon Zeroing

What went well

Weapon zeroing, as the title suggests, allows you to zero weapons to different ranges, allowing you to shoot accurately at much further distances. This means the scope considers the fall-off of the bullet at a specific range and allows you to adjust the sight so you can still aim your reticule at the target. I.e. you don’t have to aim above the target. We already have lots of scopes available and the first challenge was deciding which scopes should zero at all and then deciding whether they should auto-zero or manually-zero. This led to a much wider discussion surrounding manufacturers and their specific traits, such as high-tech or low-tech. While the team could have just focussed on the feature and pushed it out, they have also delivered a long-term plan for the scope attachments for not just current manufacturers but future ones as well.

This is something that I fundamentally believe in – that even though we are working on a live product, we should be making decisions around the final experience in the released game. Sometimes that’s hard as it can mean that certain features are not seen in their best light on first inspection or misunderstood in the short-term by the players. But it’s my job to make sure we are working towards the final vision and the team did a fantastic job of providing a feature that is fun to use but scalable in the future.

What didn’t go so well

This was the first feature one of our newest team members worked on. As such, we made sure he had plenty of time to deliver it way before the actual release. In fact, the functionality (not the visuals) was delivered the quarter before and he did a great job. Now, in most cases, this is fantastic as it means we can focus on the experience and make sure it’s of the highest quality. In this case though, due to other priorities and this being done so far ahead of the release, it did not get the full focus of the testing team as they were (rightfully) busy checking things going imminently live. Unfortunately, this meant a fundamental edge case was missed until just before the release, which was that zeroing did not work when the physics patches for the environment streamed out.

Let me explain. When a character spawns on a planet a series of patches (squares) spawn around them in ever-increasing sizes. These patches cover the render (graphics) quality and the physics collision, with patches further away having less detail and eventually no collision (as physics is relatively expensive). Now, when you move around, the patches dynamically update but it’s not one-to-one. So, a patch you might have just been in may still have its collision as you may want to go back there and it’s more performant to keep it there in the short-term than to keep loading and unloading it. In this case though, it meant when a designer loaded into the editor to test the feature, the patch was loaded and then they moved 2 km away to test weapon zeroing and it worked. Though if a player had never gone to the original position there would be no collision and so the scope would have nothing to test against. This meant in normal game conditions it didn’t work as intended, so we had to reauthor the weapon zeroing code to test against the render geometry instead of the physics one. While this change was not major, it was not ideal as we had planned to work on other features.

What we’ll do differently in the future

If I could go back in time, I would most definitely make sure that the feature had been thoroughly tested when it was completed rather than wait for the more traditional go/no go. While it didn’t affect the feature’s release, it did have knock-on effects for future work as we had to reprioritize during the quarter.

Gemini A03 Sniper Rifle & Behring FS-9 LMG

What went well

As with every weapon we add to the game, we must always make sure it fits into the existing weapons’ matrix and offers something unique that doesn’t already exist. With the Gemini A03 sniper rifle, the intention was to make a lightweight, responsive marksman rifle with a lower caliber than its heavier counterparts but high speed and accuracy. I think the team really delivered on that intention and struck a good balance between the high-caliber sniper rifles like the P6-LR and the more assault-orientated weapons like the Gemini S71. While the A03 was a simple addition, the Behring FS-9 was a bit trickier. LMGs as a weapon category are not in a great place right now as they are outgunned by SMG/shotguns at close range and outmatched by assault rifles at mid-to-long range. This is something we are acutely aware of and have been working behind the scenes to improve. The Behring FS-9 sets the standard for what we want LMGs to be – high-capacity machineguns that allow players to suppress an area without a huge loss of accuracy.

What didn’t go so well

While we were working on the FS-9, we were also working out the design intentions for all the other LMGs so we could deliver an update across the board in one release. Unfortunately, we ran out of time on the existing weapons, although we did manage to do some tweaks to raise their effectiveness. We do plan on updating the existing LMGs to raise them to the same quality bar as the FS-9, but it does mean in the short term it’s vastly superior to the other weapons in the same category. But as I mentioned above, I will always prioritize decisions around what we want the end goal to be so that we are constantly moving forward.

What we’ll do differently in the future

We have a plan to overhaul all the weapon categories and hopefully you can see that each weapon we release slightly improves on those that came before it. With this intention, I would have added more time to polish the existing LMGs as I would have liked to release them all together. No matter how experienced you are, you are always learning and this is something I will be implementing for future weapons.

Richard Tyrer
Core Gameplay Director

Locations

Space Station Refinery Decks

What went well

For this release, we were able to introduce refinery decks to some specific space station locations. These spaces are themed around the mining and refining gameplay loops and also serve as a location for future gameplay opportunities. Inside the refinery deck, we created a space specific for refining and processing along with a shop and guild space below.

After seeing the response to the cargo decks and the new space station interiors in Alpha 3.11, we agreed with the comments about players wanting to see more of a visible connection with the exterior to physically understand where they are in the station. Even though we were quite far through developing the refinery deck interior, we pivoted and adapted the space to include the mini viewing deck by the elevator lobbies. Visually, we had wanted to explore a space station experience more focused towards industrial activity for some time, including the global composition of the station to the hot and noisy interior.

What didn’t go so well

It was a shame not to see specific NPC loadouts release with the refinery decks or be able to expand some of the specific usables. However, these are all planned so hopefully we will see them come online soon.

What we’ll do differently in the future

As mentioned above, we introduced the viewing deck quite late in the process, so we could have designed a superior space with this in mind during concept and whiteboxing.

Stanton Planet Improvements & Polish

What went well

Continuing to build on the planetary improvements made throughout 2020, this was the first opportunity the team had to introduce the new and improved workflows developed when creating Pyro’s planets and moons to Stanton.

Improvement to the workflow included having the time and focus to go deeper on the global painting process. For planets like Stanton I and IV, the global painting was completely redone to be both more accurate to temperature parameters and to have a better blend and distribution of hues. As a second part to the painting, all object presents and distribution rulesets were done from scratch. In general, the focus was to do more with less. So, rather than use many types of geology all in the same space, just use one or two that work really well together. The end result is something much more realistic and natural. A good example of this is on Daymar. Another area we wanted to improve was the increased use of ground scatter objects to complexify the terrain read. We combined a mixture of geology assets like plates, scree, and small rocks with the distribution of the geology packs to give a much more integrated read to the scene.

Additionally, some outstanding geology packs were converted to use the organics shader and were processed correctly through Houdini as part of our pipeline.

Some new tech features came online during this release that we were excited to take advantage of, the first being terrain displacement which replaces POM. So, now you can actually see the terrain geometry get tessellated and displaced giving actual depth and more complex intersections with objects. The second feature is biome accumulation, which means assets can procedurally receive a thin layer of material on their top surface depending on the biome. For example, sand gathering on the top of the rocks on Daymar.

What didn’t go so well

As we were trying to close out the year and hit the Alpha 3.12 release, some moons were not able to be taken to the polish level we wanted. Also, as part of introducing our new workflows and methodology to the Stanton system, we noticed the visual styles between some moons are becoming too close together and we’re losing some diversity.

What we’ll do differently in the future

More assets will help reduce art fatigue and improve visual diversity, so moving into this year, we’re investing time and energy into more asset packs.

Stanton System Spacescaping

What went well

We were all really excited to be able to showcase the gas cloud tech as part of the PU in Alpha 3.12. Lots of hard work was done by many teams to create this new feature, so the process was about establishing a team dedicated to creating content for Stanton, and then starting to look at what we could do for the Lagrange points.

Considering this was the first release version of the tech, I feel we created a variety of visual scenarios to show the potential of the tech.

What didn’t go so well

As this was the first release, there was obviously a lot to figure out in terms of performance, and there is a great deal we can do in terms of pipeline refinement. There is also some visible noise in some lighting scenarios that we would like to reduce in the future as performance allows.

What we’ll do differently in the future

We are already improving our development workflow and looking at ways to improve the process. We are exploring how we can tie together the background spacescape into more mini-system-based forms, which then leads to smaller, volumetric gas pockets. For future gameplay opportunities, we’re looking at encouraging risk/reward gameplay inside the gas pockets with elements like lightning, radiation, temperature ranges, and flight handling.

Ian Leyland
Star Citizen Art Director

Technology

For Alpha 3.12, the Graphics Team mainly focussed on improving stability and fixing bugs with the various graphics features utilized in the release. Many of the bug fixes related to the introduction of gas clouds, such as fixing a visible dither pattern when the sun is obscured by a cloud and preventing gas clouds and particles from clipping inside spaceships by improving both the volumetric culling and particle culling systems. Such issues were expected but largely unavoidable because, although the tech has been used extensively in the development of SQ42, the artwork and scenarios are quite different in the PU. Plus, the sandbox nature of the PU and extensive testing it receives meant many previously unknown issues were discovered or raised in priority.

The team also managed to resolve dozens of other bugs ranging from popping shadows to over-bright camera exposure when a planet is streamed in. The proportion of time spent bug-fixing compared to new features was higher than we’d have ideally liked, but there’s always an emphasis on stability and quality at the end of the year and feature work has already resumed, so this is not of concern. Despite the slowdown in feature work, we did manage to maintain good progress on the new Gen12 renderer, which will be our primary focus for early 2021.

The Physics Team worked on the volumetric soft body prototype as well as the related rendering of volumetric deformation. Moreover, various optimizations were done in physics. For example, we improved the threading of various subsystems, added faster spatial grid queries, removed contention accessing local command queue, and removed contention for actor/living entities step functions (improving the living entity step performance by a factor of 2x – 5x). We also implemented a better way to pre-physicalize the planet terrain patches used for collision checks. With regard to collision detection, we also fixed a longstanding issue that could introduce additional ghost contacts far off from where the actual contacts were being processed. Furthermore, improvements were made to event queueing. The first draft of propagating physicalized shockwaves was submitted and box-shaped physics grids and bullet drag were added. SDF support was improved and research started on improvements to the setup of touch bending vegetation.

On the renderer, we continued with our ongoing Gen12 transition and related refactoring. For instance, we added a parameterizable feature set for the deferred pipeline, implemented per-object resource set updates (including RTT update for brushes) for Gen12 scene-rendering and legacy pipeline state caching (to save DX API calls while we’re still fully transitioning to Gen12). In the shader system, we cleaned up all shader reloading code, which will improve shader editing during development and give a much-improved response when changing system spec settings (e.g. graphics settings that require the use of different shader combinations). We also started a larger refactor of the shader cache backend system as it’s quite outdated and a constant source of grief with regards to maintenance and Gen12 fitness. Several optimizations were done in the renderer code. For instance, the way material constants are uploaded to the GPU was simplified and optimized.

On the graphics side, various fixes for depth-of-field were provided. The hair shader received several improvements, such as the ability to disable specular highlights on eyelashes, improved boundary occlusion at hairlines, support for ambient lights in forward shading, as well as improved hair quality during camera cuts. Dual lobe approximation for the skin shader was improved and the eye shader received a couple of improvements as well. As far as atmospheres, clouds, and the unified raymarcher go, the improvements mentioned in the previous postmortem are now available in Alpha 3.12. With that out of the way, most of the time was focused on volumetric cloud rendering. The initial draft of the cloud renderer was implemented and work on volumetric cloud shadows made good progress. Work will commence on improvements to local cloud shaping. Note that there’s still quite a lot of work to be done before release.

For the core engine system, we implemented a dynamic zone culling path in the zone system. We also fixed a few view distance-related culling bugs to do with pixel-sized objects that went into Alpha 3.11. People have already noticed that they can now see players over 1000 meters away instead of just a few hundred or so. A lot of bug fixes and improvements were provided for vis-areas, such as a fix for streaming meshes for animated vis-areas and the ability to add vis areas to CGA joints. The entity system received several improvements and optimizations to avoid unnecessary updates and searches. Similarly, the entity aggregate manager received low-level optimizations to improve work balancing and reduce memory use and contention when working with entity bubbles. There were also a few smaller improvements made to the entity component update scheduler. Radix tree culling logic has been improved, with its threading logic adjusted to reduce contention and SIMD culling implemented to check up to four bubbles vs objects per node. With regards to performance, progress continues on the engine profiler, which saw a lot of enhancements. Work on a real-time sampling profiler based on event traces will commence soon. Various optimizations were implemented in the entity system, component updates, and zone system. Based on telemetry from the PU and PTU, we continued our ongoing investigations into memory usage. As such, the engine-wide memory allocator and memory tracking system, including its toolchain, was substantially refactored and improved. To provide an additional performance boost to our servers, the Linux DGS was switched to a monolithic executable to allow the compiler to generate better code (thread local storage access in particular). As part of our initiative to build a performance regression system, we added a stress test for object container streaming too.

Regarding crash-handling, we now capture a hex stack of the render thread and embed it in mini-dumps that you (optionally) send to us in case the game crashes. This allows us to recover the full render thread call stack during postmortem debugging without the need for third-party binaries (that might be part of call stack or the video driver) to fully unwind the stack. This saves quite a bit of time as we don’t have to download the various drivers that players use.

On the animation side, we fixed code so that character blend shapes and the dynamic lighting rig don’t switch too late at every camera cut when rendering cutscenes.

Lastly, C++ 14 support was enabled for the entirety of the client server editor and relevant tool projects.

Online Tech

Load Balancing Test Framework

The pestilence warmer for Alpha 3.12 received major updates. First and foremost, the warmer now uses the new JWT identification system that allows it to fetch many tokens for impersonation purposes very rapidly. This has 10x the throughput of warmer instances we can run at the same time.

A major subsystem was added that enables the warmer to connect as a service to the diffusion gateway allowing for executing load scenarios that coordinate both as a client connected through the hub and as a service on the diffusion network.

Backend Concurrency Improvements

We were able to increase the performance of the variable service, loadouts, and the main persistence cache service. The stability of the backend increased greatly, surpassing the performance and reliability that we had in previous releases. Our low-level networking code was updated to improve both performance, scalability, and robustness. We also made several fixes and optimizations to the transaction service, rentals, and our entitlement processor.

Unified and Centralized Logging

With our new unified centralized logging system, all services send formatted JSON messages to a centralized Elasticsearch database. Each log event is tagged and dynamic data such as account IDs, player IDs, etc. are extracted into named fields, which makes searching for events or specific fields – such as an “AccountID” – very efficient. This allows the devs to easily access logs from a centralized place and track complex messages and events happening between multiple services.

Persistent Tech

Entity Creation & Spawn Decoupling

In preparation for persistent streaming, the entity creation process was decoupled from entity spawning. This allows us to “seed” the initial state of the universe into the persistent database by creating all dynamic entities prior to simulation. DGS processes will then stream persistent entities (spawn/despawn) from that database during simulation. This is an important steppingstone for a fully persistent universe.

Parallel Spawn Queues

As an optimization, we introduced multiple parallel spawn queues on each game server. This allows us to spawn multiple distinct locations (such as Lorville and New Babbage) in parallel on separate threads on the same server. Previous releases had a single queue and therefore (in this example) we wouldn’t start on New Babbage until Lorville was fully spawned. On busy servers, this can really reduce the wait time in some cases. For example, when spawning waves of AI ships or respawning in a hab.

Network Tech

Time & De-Syncs

How the engine measures the passage of time underwent a complete overhaul. Accuracy was improved both in the measurement of time and in its synchronization between server and clients. How the engine uses time to update its logic and physics simulation was changed to eliminate errors that could result in simulation time passing differently on the server and clients. Many smaller bugs that had caused timing errors to grow on long-running servers were also fixed. The network synchronization of vehicles and physics objects were updated to take full advantage of the improvements. The accumulated result of all these changes was a significant reduction in latency and desynchronization issues in many areas, even under harsh test conditions for network and server performance. Besides improving the overall player experience, this work was a necessary step towards server meshing, where simulating the game across multiple game servers would have made desynchronization issues due to timing errors worse.

Authority API

In preparation for server meshing, the team performed a sweep on the remaining tasks to convert code to the Authority API. Over the last 12 months, there has been a coordinated effort by all teams to update the game-end engine code to this new system. Thanks to their work, a large majority of these tasks have been completed. With a concerted push, we’ve reduced the number of remaining tasks into single digits.

Connection Flow

In a server mesh, a client may connect to many different servers during a game session. Part of the work towards server meshing requires separating the process of connecting a client to a server into distinct stages. These stages can then be executed independently without requiring a client to completely abandon its existing game session. Significant progress has been made towards this although there is more work to be done.

Marco Corbetta
VP of Technology




See you in the ‘Verse!

We believe that giving this level of insight into our development process is highly valuable, especially when you can read the thought processes, reflections, and learnings direct from our senior developers. As mentioned last time, we’re committed to transparent development and will continue to provide quarterly patch postmortems going forward.

End Transmission

Comments
063.0

Feedback

Loading Additional Feedback