10 for the Developers: Episode 05

Transcript — every row opens the video at that moment

1. 00:00:07

   [Music] hey guys my name is Ry Vasquez an associate producer for cloud Imperium games um we're here for 10 for the developers and for the first time we have Vincent Sinatra QA lead nice so we have questions I want to also thank the subscribers for making this possible it's really awesome that we get to kind of like directly interact with you guys as well as take questions and then kind of we'll give you our opinions our suggestions or you know kind of our ideas on and thoughts on what you guys are asking so ready to get started oh yeah let's do it sweet so the first question is from Solitude for both what aspects of the gamer process by which I mean setting resolution visual quality

2. 00:00:55

   audio levels control schema Etc do you find frustrating would you change given scope and and resource to do so for me that answer is easy it's the holot table what everybody loves the hollow table really who are these people I don't know I personally don't really like the way it's functioning currently uh I don't like to look at it don't like to test it in fact I got some new testers now we're we're training up and I'm making them test it because they're the nuggets and that's their job I guess personally like um the party system right now just like ah the party doesn't bother me but at the same time is I know that these like these things are being worked on and like a lot of the feedback from the fans we've taken tons of feedback on things and so we're definitely working on it so we're pushing that soon I cannot say win so do not ask when don't thinking about even asking win don't do it yeah pretty much

3. 00:01:44

   the next question is going to be from a Monti I think that's how you say is that how you say yeah I guess sorry what sort of techniques or procedures do you recommend for those of us that want to hunt and Report bugs to cig how can we most efficiently go about collecting the information that you need so this is kind of a question for you yeah I believe this is probably a QA question and issue counsil type question uh so basically what we PR primarily want is you know people to be you know concise clear uh clear steps for any things that you're sorted doing but also include you know sometimes you're only reporting the thing that you saw and what you were doing when you saw it but sometimes what you were doing before that impacts things like we had certain situations where people were getting disconnected while joining Crusader they would just tell us hey I got disconnected joining crusader but you know what it was because they were

4. 00:02:31

   joining Arena Commander before they transitioned to Crusader and their tokens weren't getting cleared and that was the issue so as much information as possible as they can provide is usually the best for us because then we can pick through all that information and and you know get really good steps for the developer so that they can actually get a fix in so I know with like release notes and stuff we have I know do you you guys set the release notes or so maybe like in there we can be like hey guys can you focus on this like do we do that right now we don't do that right now it's actually a bad idea you know if somebody was a producer they might be able to push something like that forward you know that might be a good idea going forward man see this is what happens around my mouth yeah exactly all right next question uh I guess this is also for me nurila since thanks to the issue Council

5. 00:03:19

   players are somewhat involved in the QA process do you search the council frequently in search of hints to better explain what went wrong for a particular problem or do you just wait for issues to receive the appropriate number of votes and then react um well me personally uh since I'm here in LA and we got a really small team uh you know mostly we're embedded testers uh we do a lot helping the developers and stuff like that I know our Austin and UK we got great QA teams out there who are always in the issue Council making sure that they're checking issues now a lot of those issues are already written up in our database because we ran into them during our normal sanity checks and smoke tests and sweeps and stuff like that but the ones that do get rise to the top those are the ones we investigate we make sure we get in the database we make sure we get fixes out there because we know the backers want that stuff fixed as soon as possible so some people might not even know what the issue council is what like issue council is where the backers and all our

6. 00:04:06

   subscribers should be going to report any sort of issues and then if you run into it you vote it up and then the higher it goes up in the chain the sooner we take it we take a look at it and get it taken care of nice that's kind of cool I don't think I've ever used it since I just like go over to your desk and I'm like yeah and you say is this happening and then you hit me yeah that's exactly what happens pretty much for everybody poor guy hasn't put up with this yeah next question from Drew drug drug drug h i I guess yeah let's go with that one so how can we support testing better should we just try more crazy stuff actually intentionally cause Havoc to break things does the wealth of PTU data provide any new insight into the process and how does one Prov prepare to QA the sheer number of future features while simultaneously changing the engine adding new parts increasing complexity I

7. 00:04:55

   feel this must be a moving Target for QA in an unstable environment yes yes yes it is definitely an unstable environment in fact more days than not it is quite unstable there's going to be certain areas of the code you can't even look at uh as new new iterations come down um as far as causing Havoc well when you're out there in the PTU I think we want more controlled Havoc actually yeah we were say controlled Havoc or like kind of like planned chaos planned chaos that's the one that we were kind of chaos that's what we want um because you know doing strange and weird things is always great I mean sometimes you find the best bugs by doing something that the average user would never even conceive of like what happens if I land this ship on top of this ship on top of another ship you know and you end up with some sort of ship setion going on or something like that uh you know there sometimes you can do that sort of wacky

8. 00:05:43

   stuff and you can find some really good issues but you don't want to do that to people who don't know what you're up to then it's just griefing basically at the point like don't don't grief people don't be dicks right just be nice to people and like if you want to test something crazy seriously just we talked about this like just hit people up and be like hey just ask hey guys I want I want to do something crazy is anyone down for crazy usually people be like oh crazy we're in we're doing thisly we got we had a pretty good crazy base uh going on and you know people just like to say Hey you know hey I'm going to try something wacky does anybody else want to come out here and help me out with this and you know usually you'll get one or two people to jump in I mean I know I've done it in the live you know server a couple of times uh you know so that's one of those things that you can do as far as you know Havoc um as far as the PTU and how it helps us with the process it really helps when it comes to Performance and stability we get those metrics from you know hundreds thousands of users and uh

9. 00:06:32

   that's obviously something that we can't do internally because you know our test teams are only so big and also on that like people need to just log in and just be there cuz even just logging in for a little bit here and there and stuff like that we kind of put the service to their Paces the engineers are getting really good information the designers are getting good information because like the designer can design something and then all of a sudden like it works when you have like one or two people there you see this all the time probably like five people there but then as soon as you hit the ninth person or the 10th person that's when like everything just like breaks down like crazy you know whenever you can get those numbers it really helps because again the QA teams are kept small and focused so that we can you know know lasery accuracy on what we need to be checking but you know so Point takeaway is if you're going to have if you want to do some like you know crazy stuff seriously just put it in chat and be like hey guys I'm trying

10. 00:07:19

    this out does anyone want to join me ask permission before you start you know using a Glade to lift up other ships just don't kill someone cuz like we've seen things on YouTube where people are just going crazy like they just like Ram another ship like a person's not even like taking off on like oh come on you know that that's that's griefing because you also don't know how long that person has to play and they might just be in you know you may be in there for like a 4our session trying to hit up as much stuff as you possibly can they're only there for 15 minutes they you know they got their kids they got to put change diapers they got to get the food ready they got a bunch of other stuff they just want to jump in and see what we have on the next PTU patch and they don't necessarily want to help you do crazy stuff so you know just clear it ahead time if you're going to involve other people in your wacky Havoc causing tests the thing is we have such a really strong community and like we don't need stuff like that

11. 00:08:07

    to like kind of just bring it down so yeah kind of I don't knowa contr controlled and planned chaos is where it's at so next question would be logical chimp when looking at the QA team workload roughly what is the split in testing between running through predefined scripts regression testing ad hoc experimental testing targeted specific feature testing EG Gathering stats on ship performance and as a followup what determines and changes this allocation and what determines if you need to increase time on regression or spend more time ad hoc testing uh well as we've mentioned it's largely a moving Target um you know usually we have our set predetermined tests that we're going to do on every single build be it a smoke or sanity check or something like that and whenever a new feature is developed of course we're

12. 00:08:54

    going to you know go through a gamut of tests that are designed around that feature but we're also going to ad hoc areas around that future uh you know as it comes out I would say was probably about 40% regression 40% targeted uh 30% ad hoc because you know QA we give 110% every time 110% that's right so actually we were talking about this earlier cuz like whenever like bugs come in and like so we have like a little thread that we in production is like hey these are being flared out and it's mostly just production and keyway that are on these threads and it's basically like hey this is the latest blocker this is the latest thing whether it's found through the community or whether it's found through just us internally testing and all that stuff so like these are getting major priorities so like as soon as these are sent out the developers get it and then once the developers fix it like a producer is basically hitting up

13. 00:09:42

    like one of the leads like you and just like hey we just fixed this here's on and then like people will just change Focus like that and just be like all right we need this we need this now we need this now so yeah it's like it's like like building in quicksand Almost Pretty the situation is is always fluid from one day to the next from one hour to the next cuz all of a sudden somebody will come with something new and that all of a sudden takes priority and everything else that you had get shifted down a rung on the ladder and you have to get to it later yeah it's always a lot of fun fun is not quite the word I actually yeah no it is fun it is fun it's a lot of fun it's a lot of fun all right so next question is from Steve Hunter when doing QA testing past resolve bugs are often retested to make sure that new fixes don't cause them to occur as this

14. 00:10:30

    would make the test list grow uncontrollably Qui large quickly how do you decide when to no longer retest an old bug and thanks for all the good work thanks Steve yeah um this is actually a really good question uh so when it comes to that usually we focus on areas that have been affected by code recent code changes like you know recently Mark and call came me and said hey done a bunch of changes to landing gear so that immediately tasks us out to start you know focusing do checking all those old landing gear issues to make sure that those don't recur uh because you know of the changes that he just implemented to make sure that previous fixes didn't get stomped on by new change lists and so on uh so I mean that's pretty much we try to stay focused on the things that are in motion and are still changing uh and then if you know something hasn't changed for after build after build after build after build well we're not

15. 00:11:17

    necessarily going to be looking at all those old issues again we're going to you know if they do return we you know we will reopen them but you know we don't necessarily focus on them yeah and that's the thing is like I know once the bugs have been through their Paces they've been resolved fixed and like you said once it hits a new area like if we see it again then we'll check the database and being like Oh okay this is back and in all honesty like there's been some bugs especially like with landing gears in particular oh yeah the landing gear comes back that's why I brought it up because you know poor Mark has been you know banging his head against the wall with those sometimes I mean I swear to God some of these bugs spend time in the Lazarus Pit and then they immediately come back and they rise up stronger than they were before it's just unbelievable with some of them it is pretty crazy far Seeker I know you you've posted some about this in your feedback threads already but could you give us an overview on how this activity especially anything that submitters can

16. 00:12:04

    do to make the feedback more useful as well as more details on how this feedback will be communicated to you and used to the rest of QA and the ship designers um yeah so basically uh what I'm trying to do is because it seems like we got so much feedback I mean I had pages and pages and pages of feedback which that was pretty awesome by the way was very awesome I mean and the community is very passionate and they gave us you know just tons of information uh but it somewhat was information overload for you know developers and designers who are already working really hard on new things and they're like you know like you know when am I going to get time to parse through all this data uh so moving forward we're going to try to slow down the rate of those so that you know we give them more time with each ship uh for the most part it's still in the works like when we're actually going to be doing these I think right like yeah we're still we're still breaking it out like I said I was I was you know rapid fire machine gun style

17. 00:12:51

    getting that stuff out and then obviously that was a little too much so we're going to try to slow it down a little bit ous rambunctious I might add yes yes yes a little bit a little bit you know overeager I'm kind of new here trying to make my mark you know I got I got I got impress guys like Randy every other day you know so you know I was trying to work on that but uh you know um as far as as far as what um details you know just keep things concise keep things clear uh try to keep things within scope I mean we're not always looking for you know hey please buff my favorite ship type of stuff um you want to make sure you know exactly the type of things we're looking for perhaps you know on my part I need to be a little more clear as to what we're asking for and we had some Growing Pains through some of the threads with that because the thing is some some of the stuff like we were talking about earlier is I'd love for people to break down the feedback from

18. 00:13:40

    different sections so like even multiple post so that way like hey this is the Arts post on this so that way you give everything for art break it down to design mechanics like break it down as much as possible like yeah balance wise like all right so I'm flying a 300 I'm flying a 315 they're almost the same ship but not really like but then kind kind of just breaking down that thing of like as much as not don't be as verbose like clear clear concise definitely you know but don't be afraid to give more example or give like at least cuz that those the examples you guys give as well as like just the detail that you guys can give us anything is useful yeah and and I was pretty much amazed at the detail I mean cuz we had like Graphics you know particularly for the 300 you know people were like hey this is how

19. 00:14:27

    the redesign work and I'm like is this is this is this a resume is this a job application or is this actual just you know normal feedback you know so I mean the detail was crazy good in some regards uh and then in others you know some people are all like you know hey you know give me a buff you know for the most part I want I want make this sh I want five gimbals on my ship or something like that you know those we don't date who does it you know uh but but for the most part you know uh as long as as long as it's clear as long as it's concise you know and uh you know keep things polite and a thread somebody says something you don't agree with you know hey we're all trying to make the best game together you know that's the great thing about our community is you know working together putting on that thing is like if you guys have different varying opinions that is perfectly fine we actually welcome that because then we see different perspectives on how people

20. 00:15:14

    are looking at the designs how people are looking at the ships the role of the ship everything like any feedback is good feedback we can't always act on everything and we're not going to be acting on it like instantly so give us time cuz we're still working on things we only have so many people to actually do everything yeah and uh I'm going to try to make it you know easier for the devs and designers to digest when it does come in you know try to get like a top five bullet points and then you know here's the rest of the report when you guys can get around with it but these are the major things that are really either bothering the community or that they're they're really excited about getting implemented for their particular favorite ships be super useful yeah very useful all of it all righty our next question is from Aragorn BH since you have played many hours of Star Citizen yeah yes and know the game mechanics so well I'm still working on that what

21. 00:16:03

    plans do you have for yourself in the Pu as a gamer so this is kind of a loaded question and the reason why it's loaded is because I really want to have kind of two personas kind of when I go out there like I want to do one where I'm like a mercenary SL pirate and stuff like that and seriously just get in on that like whole clandestine thing and like just be on that black market I think that would be hilariously fun cuz I never play that type of character I usually every time I play an online game I'm usually either support or healer and then of course my other one that I would play would be like a minor like when I played Eve I was all about being a minor and a Trader so I think like that would be really cool so like just doing some mining doing some cargo stuff that's why like when I was giv the ship the the caterpillar I was like all about it just because it had that different mix of things that I was like I could do this I could do this so I'm actually looking

22. 00:16:52

    really forward to just I don't know trying that out it's going to change I guess well yeah know I mean everything's like you said moving Target but uh yeah know that's pretty good uh I know I once the game's out I'm going to have one mission in mind and that is to find Randy Vasquez and kill him pretty much well yeah I mean I like hunting you guys down I mean that's the most fun of the day you in play test completely destroying you guys in play test is I so much fun stand you so much in so once we're actually out there you know I figure I could be some sort of cig Bounty Hunter and I'm just going around letting you know other people know where cig people hanging out maybe I could do some screen hacking you know and then you know hey guys they're over here and then you know we swoop in and destroy you all I think we did a play test what is it like a month or two ago and then basically I think it was Mcall

23. 00:17:40

    or it was EI it might have been you I think was like by the way guys Vasquez is at for Korea I'm missing a know like I had like four or five Ships coming in and I'm just like oh God and then I'm trying to hide from everybody I shot a few people but then I was up shooting like I think I shot maybe that's what I'll be I'll be an information broker who then leaks out where C's people are and then people can go hunt them and have fun doing that nice nice I could probably make a little you know a little bit you know hey QA salary is not exactly that great you know good try all right so next question from space is nice I think this one's for you uh what is the priority of developing things is it ships places economy what takes priority over other things like shipping over ship buying or more to do in one place over more places to go thanks

24. 00:18:29

    all right so as we've stated a few times like it's development is always kind of a moving Target right and priorities shift daily if not sometimes hourly based on what's broken or what the fire is and stuff and because we want to give you guys a stable game yeah we want to give them stable game we need to give you guys also in QA a stable game because you guys can't test if it's not stable like you guys testing it unstable environments doesn't help us because we don't know if the bugs you're giving us are actually there so we can't help you because you know somebody will be like hey will you check this and was like well we can't check it because yeah so kind of to go off the questions like versus ships versus levels versus economy pu mechanics and systems we have so many people around the globe working on this game M and everyone has their own everyone has their own Focus

25. 00:19:18

    everyone has their own priorities so we have an overall Global organization and then we have within that organization we have the different Studios then within that we have each individual teams working on something where they're their designers their artists QA production Engineers audio narrative you know like we have so many different aspects of this that everyone just is all over the place all the time there's no ever a priority one thing over another like for instance um if Crusader is broken right then that's kind of a level design but it's also design but it's engineering not so much art so then that becomes a Focus for the select people where as where the artist will still be working on the stuff that they're already working on so like they don't they don't

26. 00:20:05

    change Focus now if a ship is completely broken and a ship is like needs to be redone due to Vis areas whether they need to be you know lods ship set of issues then it's like the tech designers and the artist they're shifting over to help with this and sometimes they need engineer support so it is a case-by case basis on really what the priority is and that is actually coming down from the community cuz every time the community like they they're they yelling for something they need something they want something you know like we're taking that into account also the leadership team of cig is also like hey you know so like they're kind of always playing the balance game of all right this is what the community wants and this is what they need this is the leadership team wants and needs and then kind of just doing that balance game like this is our priority for right now and it's our

27. 00:20:52

    priority until something else changes and you know but we still have like schedules and everything else we're working on but still have planned future rolls stuff like that as well it's just like you said you know things kind of change up Midstream every now and again so it is what it is last question is going to be from Daz and he says hi Randy hi um can you share some or both of your own goals and the company goals in production this year is there a particular area you love working on and are keen on improving and what game features are we likely to see in the short medium term for this year cheers stz so this one I kind of for me myself I just want to survive as a producer like it's it is so different from becom from a designer I'm enjoying it it's completely different I'm really loving I

28. 00:21:40

    like the communication aspect of it yeah yeah you're really good at communicating really good at communicating got an inbox full of your communication mhm so I love that and I love facilitation so basically making sure the team's taking care of whe if you know go and everybody make sure they have what they need hey what's up what you guys back rubs regular back rubs tell come Randy next you know my numbers the bathroom wall thanks that could be a good and a bad thing so um I'm really enjoying that something that I want to improve on though is I really want to get the ships pushed out yes in a better kind of like timetable I know um we're we push ships out and we're like letting you guys know when ships are hitting and each ship is its own like little special snowflake

29. 00:22:28

    so it's been kind of hard to like kind of nail that down because each ship has its own mechanics each ship has its own features you know and there's those features that are tie in all the ships that need to be you know pushed out stuff like that you know like you said the the Shields and armor balance that you know the KX and everybody's been you working really hard just finish that audit for the AIT for the shields you had to do all the T yes I had to do all the tests too but uh you know um trying to get those all out would be really good and you know help you know give us more of a final picture of what the game's going to be like you know eventually uh you know CU of whereas like when with each ship that comes down it's just you know all right but that's just that's just a a small piece of the greater puzzle and that's the thing is the audit you just worked on as like part of something bigger too cuz we're doing we're redoing some of the Shields and how they work just because it's going to help optimize the game and how how lasers affect the shields how

30. 00:23:15

    ballistics the sh all that stuff like is just being taken into consideration so like the Design's changing a little bit and it's changing for the better and it's changing to make sure that you know players are given the proper feedback they need as well as like it has all the functionality NE Necessary Things it has to have like I mean it's just Shields are just crazy integral to the whole thing as a as a game yeah no I mean especially for dog fighting even especially for dog fighting well I mean because it's hard to balance the ships against each other when that system isn't fully fleshed out just yet were you around still when they had that like the constellation and the Hornet bug like yeah I was one of the ones who it was like the Hornet constellation were using the exact same Shields so yeah no that was that was that was a fun one yeah so there's so many things with shields but yeah so that's like an example of something that you know we're working on that is going to be coming

31. 00:24:04

    soon yep cool well I think that was all 10 wasn't it yeah that's 10 for us all right so that's uh 10 for the Developers for us today uh just once again want to thank all the subscribers you know and all the backers everybody makes this possible for us to do this uh I know we may be quite weren't quite as entertaining as Tracy and we trying so we would like to come back maybe if you know if you guys are into it uh you know they're into it if they're into it you know of course everything has to be consensual but you know if if we could come back that'd be nice awesome I always love talking to the community I always I always like this opportunity this is definitely something new for me and I am enjoying it and you know hitting the ground running I hoping we'll communicate later we'll communicate later yeah great thank you more communication

32. 00:24:58

    hey guys thanks for watching um temp for the chairman uh if you guys would like to uh see more episodes go here if you guys would like to subscribe to our YouTube channel and always keep up to date with all our video content go here and uh if you guys would like to watch episodes of around the verse go here please and I will see you in the [Applause] verse