basod @basod13

A post I've been wanting to write for a long time is about developing software at scale at Riot (at least on League Gameplay). There's a lot of implications due to our size that adds a lot of unique challenges. There are often suggestions like: "Why don't you do 'X idea that seems easy', but in reality is a lot more difficult than it seems. And sometimes these are great ideas that we should do! Sometimes these aren't so great ideas though... eg. "Why don't you do this thing!" which solves that problem, but has some massive downside, or would in reality take 2 years to build at scale. Tackling these challenges is fun, but simultaneously difficult. I wanted to dig into some of them, not as an excuse for why we can't do/are slow to do some things, but mainly because it's fun to talk about/to help aspiring game developers/transparency for how working at Riot (at least on League Gameplay) functions. 1. Size - Let's just say as a lowball estimate, there are a few million lines of code in our codebase, thousands of product guidelines, design principles, art style guides, quality standards, etc. - Let's say any individual could realistically be an expert (you know the most about this thing) in ~5% of your area (eg. Design/Engineering, etc.) and be a contributor (you know enough to be able to positively contribute) in ~30% of it. - Simultaneously, the code/knowledge base is constantly being worked on/improved, so your understanding of what that system was/is doing is changing over time - There are definitely things across all disciplines that aren't up to modern standards, but even the best written game foundation is so large that it's unrealistic to know all of it - Because League is a big game, there are also a lot of protections that need to be built in/designed around to protect us from being a target, to relentlessly optimize software performance that would otherwise be totally fine for something at a medium scale - There are problems that only get surfaced when things reach a certain size, which is a great problem to have, but means that the experts need to be even more specialized at resolving these problems - One inefficient table read or service query could rack up to significant performance/monetary hits when this is multiplied by millions of people triggering that operation every day. 2. Specialization - People have areas of expertise, whether you're a Systems, Mechanics (eg. Champions), UX, Progression Designer, Rendering, Profiling, Services, Front-End Engineer, etc. - In the same way you wouldn't ask Messi to play goalkeeper (even though he probably could), some things can be very difficult to get movement on without the right person with the right expertise - What on paper could look like 7 Designers, could realistically only be the output of 3 Designers if they're not working on the right project for their skill specialization - Implementing something the wrong way, or making the wrong design choices could be potentially devastating; we have processes in place to make sure experts see things before they go in, but even then, mistakes can happen - Definitionally, there are always far less experts than there are contributors. The more an expert is reviewing, the less time they're spending on actually implementing and doing stuff that only they can do 3. Teams and Organization Management - At Riot, we operate within team boundaries - Teams own certain things within their area of expertise and have Rioters with various specializations to work on said stuff, follow up on broken features/bugs, make new things. - Even if you have the right specializations, there's also the institutional knowledge that has to be reckoned with. eg. If we hired Faker to work on our Live pod, he would not be performant right away - Playing and designing the game are 2 entirely different beasts and there's 15 years of past design decisions that you have to reckon with to be baseline performant - When priorities change, it can sometimes mean that the specializations on those teams no longer make sense, meaning either the priorities can't change effectively, or search needs to be undergone to other teams to find individuals to service those gaps in specialization - If Rioters move around to help a team on a gap, it can also mean that the thing they were originally working on falls behind - Some tangible examples are when we moved Designers around to help get Arena's first release over the line, carrying delays in work that Summoner's Rift Team or Champions team were able to do - We evaluate the tradeoffs of what these moves would mean, alternate ways of solving the problems and decided this was worth it - When it comes to finding which team should work on a certain thing, that is also a major challenge in itself. You need to know the right teams, have good relationships with other leaders and people to ask and know where to look to find which teams own different things - As an example, if we noticed Fog of War was suddenly non-performant and tanking our server performance, it could be a number of different teams that could theoretically solve that problem. So it comes down to negotiations between teams for which Rioters have the best specialization set and how that would tradeoff against the opportunities that they're currently working on, which can get quite complicated and takes time, but has to be done, because it means we can get things out to players faster and at a higher overall quality An Example - So to use an example of "make players play 5 normal games before playing ranked" that I've been hearing recently. A great idea and one we've been discussing internally. To make this happen though, you'd want: - Ranked team to propose the design for the change + the edge cases, the rollout plan. The ranked team would also consider whether this solution is the best thing to do vs other potential solutions that have different value, dependencies and costs depending on our staffing and skill specializations available (eg. improve our seeding, so it wouldn't matter what queue you played -> does this get our seeding algorithms to the quality level we would want) and has no Client/UX work related dependencies - League Data team to calculate the correct number of games, how this interacts with seeding models, etc. - One of the League services teams that handle ranked services to figure out how to calculate, store and query the information in the most efficient way, also future proofing - Client team to understand how to query and pass the data at different areas of the client - UX team to understand how the flows work (eg. do you prevent inviting from lobby if ineligible, where is the warning that says you can't queue together/what does it look like [is it on the queue text itself, or in the lobby], do you show the warning when they're in lobby and grey out the queue button, etc.) - Client team to implement and handle all the edge cases of the UX design - Build and Release teams (a lot of this gets automated) If all goes well, despite the large number of teams involved, these dependencies and negotiations of work can happen pretty smoothly and we might be able to get it out within a few months, but for example if any particular team are all hands on deck for a higher priority project (eg. if the UX team was fully wrapped up with working on something like Season Start), then it can mean that this particular solution is a non-starter. What on paper seems like a pretty simple idea, when it's broken down into its constituent parts can become pretty daunting to approach Hopefully this was insightful, this wasn't my usual type of content, so let me know if you'd like to hear more/less of how game development tends to work!

📢Division 2 Playoffs Bracket The regular split is over in our Division 2, so now it's time to find the overall winner! Six teams have advanced to the playoffs stage, which kicks off on Tuesday, March 26👇

Fighting to the end! 💪 Tonight we showed great fortitude and courage when we took on @GardenGamingGG, but unfortunately we fell to the gnomes 🪴 This marks the end to our spring split in which the team has shown immense passion and displayed some absolutely fantastic gaming ♥️ We wish the play-off teams best of luck and we’ll be cheering on from the sidelines! 🔴 ⚫️ ⚪️ #WeAreUniQ

@SynapseSprinter @WBadeby @kvwlol @holmberglol @kanskelol @ZilenceLoL @Blodsteel @JackZimmer93823 @NLClol @uniq_club And special mention to our subs @SprayGT @jonatan78077141 @zvfaii (100 WR) for cheering us on the side.

Thank you to @SynapseSprinter @WBadeby @kvwlol @holmberglol @kanskelol @ZilenceLoL @Blodsteel and @JackZimmer93823 for all the hours you grinded into this @nlclol split through scrims and officials, you guys are true professionals. You are all @uniq_club legends!

@GardenGamingGG @uniq_club Lets goooo!

It’s time to finish off our regular season with a bang! It might not have an impact on the standings, but it should still be an exciting match against @uniq_club! twitch.tv/unluckykarvis

📢K5 in @NLClol DIV 2! 📢 ⏰19:00 CET ⏰20:00 CET ⏰22:00 CET 🎥1st & 3rd game vs 4W & 42 Gaming: twitch.tv/karolinernaesp… 🎥2nd game vs G2Hel: twitch.tv/unluckykarvis

twitch.tv/unluckykarvis Last games of NLC 2 group stage!

@danpshort messaged back

Final matches of @NLClol div 2 before playoffs and of course I need to be a part of it! 20:15CET/21:15EET @BlueWhites vs @42Gaming_ over at twitch.tv/tebz188 as always. Lets close the regular season with a banger!

Hey all - We know many of you are hungry for news about the @riotgames #MMO project, and we really appreciate your patience and the incredible support you've shown us so far. I’m writing to update you today on where we’re at. And before anyone panics: yes, we are still working on the game. #Leagueoflegends

@Colomblbl @G2Samourais WP!

We promised a banger ending to the day and we surely delivered, unfortunately though, it ends in a loss versus @G2League 👏 Weren’t able to get the snowball rolling enough until G2 were able to strike back. We’ll strike back very soon ourselves though 🥊 🔴 ⚫️ ⚪️ #WeAreUniQ

@FruityFresh5 gz

The last game of tonight in @NLClol Div 2 will be a banger as we face @G2League 🔥 Locked in and ready for yet another important game 💪 ⏰ 22:00 CET 🔴 ⚫️ ⚪️ #WeAreUniQ

📢NLC Division 2 Leaderboard We are getting very close to the playoffs down in Division 2! Some are still chasing playoffs while others are going for the best seed possible👇

Arguably FIFA also sets the gold standards in funnelling to nepotistic cronies. XD

If only we see more trickling down of this money down into grassroots ecosystems. Like the gold standard of FIFA (as a non profit registered charity) investing majority of earnings into grassroots football. Tier 1 leagues ate the brand but grassroots are our future too in esports

Adjusting our LoL Esports Strategy riotgames.com/en/news/lol-es…