Home The Bridge
Options

The Voyage Project (Part 2) (Sort of)

Bylo BandBylo Band ✭✭✭✭✭
While the first Voyage Project (https://forum.disruptorbeam.com/stt/discussion/15954/the-voyage-project-part-1/p1) was winding down work had already begun behind the scenes on a part 2. The basic idea originated with @Prime Lorca [10FH] so I cannot take credit for thinking it up, but I did do a fair amount of grunt work that needed to be done to present the project on the forum, but by the time I got the preliminary work mostly done, my laptop died and all that work was lost. A few other things contributed to a loss of interest in setting it up at that time but the idea was so profound I started to just test it on my own informally, along with Lorca and @Banjo1012 , with Banjo accidentally serving as the control group by taking the exact opposite approach!

All of this is a nice way of saying that I am not going to run the second Voyage Project, because based on our own informal observations over months I am so convinced that what I am about to say is true that I do not believe this project needs to be run. As a result of work trying to set up this project I have forever changed how I staff my voyages and I am much happier now as a result. I will still explain what the project was going to be in case people want to set it up and run it officially to get actual data. As with previous projects, I will be using the base stats for all cards mentioned (no bonuses) to keep them all consistent.

THE IDEA:

The premise is/was simple, we were trying to determine the actual impact of proficiency scores on voyages. I will keep this initial comparison simple: say you have the rest of your voyage staffed and you need more Command and you have 2 choices, an immortalized Mirror Jean-Luc Picard or an immortalized First Officer Burnham, which should you choose or does it even matter? (In this case all other concerns like other skills and traits are irrelevant).

Mirror Jean-Luc Picard (Command): 1,042 + (243-753). Total score: 1,540.
First Officer Burnham (Command): 1,359 + (88-294). Total score: 1,550.

These voyage scores the game displays while filling seats are not real and instead represent an average expected value of sorts as they are an approximation of the crew's contribution to the chosen skill. In the above example, while both voyage scores are virtually identical, they get there in very different ways. F.O. Burnham gets there with with 93% certainty (7% RNG) whereas Mirror Picard gets there with only 83% certainty (17% RNG). Essentially, F.O. Burnham gets 93% of her Command score from her base (1,359) plus her minimum proficiency value (88) for a total base score of 1,447 toward her 1,550 score, and Mirror Picard gets 83% of his Command score from his base (1,042) plus his minimum proficiency value (243) for a total base score of 1,285 toward his 1,540 score. When staffing a voyage the game treats these two crew roughly the same when sorting by Command, but do the differences in how their scores are composed actually make a difference toward the success of the voyage?

THE PREMISE:

Once Lorca and I hashed out the basic idea, we had to define a few terms relating to this project, those being Low Proficiency and High Proficiency. In general terms, Low Proficiency crew are those like F.O. Burnham with stronger base+minimum proficiency scores and lower proficiency maximum scores, and High Proficiency crew are those like Mirror Picard with weaker base+minimum proficiency scores but with large proficiency maximum scores. For the purpose of being thorough, cards like Locutus provide an interesting case as he has very high maximum proficiency scores, but his minimum scores are also fairly robust so his overall proficiency status is less obvious.

It was at this point where Lorca and I realized the huge task ahead as we'd need to precisely define both concepts with specific parameters/ranges and then generate a large table of crew fitting both definitions while ALSO establishing the methodology for an actual test, and despite the daunting nature of such an undertaking, I was many hours into that process when my laptop died. So without a strict definition of the terms but with a solid understanding of the concepts, I began to staff my voyages by actively excluding crew I perceived to be High Proficiency from my hours of compiling crew proficiency lists, so crew like Armus, Gary Seven, Bartender Guinan, Captain Braxton, Fury Kes, Boothby Replicant, Red Angel, etc, and take mental notes of those voyages' projected and actual outcomes.

I held to this very strictly for months, Lorca did sometimes but other times would send voyages we jokingly called “The RNG Express” which had High Proficiency crew stacked to the rafters, and Banjo was almost always sending out voyages full of High Proficiency crew in his quest to hit a 12 hour voyage, and we started to notice a pattern so readily apparent that it has informed all of our voyages since; voyages with several High Proficiency crew are much less reliable than voyages that avoid using them. For the purposes of this exercise, reliability is roughly defined as voyages that achieved their expected potential length, which for us was basically, “The numbers say this should hit 10 hours, but will it?”

The RNG Express model made the expected duration some of the time, sometimes zoomed way passed it, and others flamed out spectacularly early. The longest voyages we sent were the ones with the most High Proficiency crew, but they also accounted for the shortest voyages. The voyages we were sending without any of the High Proficiency crew were able to get much closer to their expected duration lengths reliably but were never much thread to do much more than what was expected.

THE NUTS AND BOLTS:

Here is how I think it works based on our observations. It is my belief that voyage hazards operate the same as away missions on the galaxy map; they have a target number your crew need to roll in order to pass, and all 12 of your crew combine their base scores for the selected skill and then make a proficiency roll based on their 12 proficiency ranges. This proficiency roll gets added to the combined base scores and this amalgamation gets compared with the target value of the hazard and a pass/fail is determined. As voyages progress these target values on the hazards slowly increases until eventually they are so high they always fail.

I will demonstrate this using three simplified voyages using F.O. Burnham and Mirror Picard from earlier. For simplicity all three voyages will only be concerned with one skill, Command. One voyage will only run two copies Burnham (BURNHAM), the second will run one copy of Burnham and Picard (BOOTS), and the third will run two copies of Picard (PICARD). Here are the three voyages' Command totals. (To help understand the table, the three stats displayed on the cards are in the primary colors, and the secondary colors of orange and green represent combining the various numbers, so yellow+red and yellow+blue.)
nykyj51saywg.jpg

(1)
«1

Comments

  • Options
    (HGH)Apollo(HGH)Apollo ✭✭✭✭✭
    Can you define what you consider the ranges of high, med, and low proficiencies? The one voyage we have that went 12 hrs had a couple high proficiency people Red Angel and Mambo Picard but everyone else seems to be like medium proficiencies. Would you concur? How does that voyage match with your findings in your opinion? Does it validate it? Here is all of the crew for that 12 hr voyage in Datacore.

    https://datacore.app/behold/?crew=emh_android_crew&crew=braxton_captain_crew&crew=dsc_spock_science_crew&crew=picard_mambo_crew&crew=janeway_evolved_crew&crew=data_scrooge_crew&crew=picard_numberone_crew&crew=bashirs_parents_crew&crew=spock_evsuit_crew&crew=leila_kalomi_crew&crew=paris_luau_crew&crew=dsc_burnham_red_angel_crew
    Let’s fly!
  • Options
    HaBlackHaBlack ✭✭✭✭✭
    edited January 2021
    I suspected something like this, but with this analysis it confirms it. Now I need to rework my spreadsheets.
    I was concentrating my efforts in reaching over 11h, but sometimes same crew combo gives me 9:30 h and sometimes 11:15 h, this is obviously beacause of high proficiency range of some of the crew I used.

    So now I need additional columns for lowest possible score and highest and to try some voyages only with low proficiencies.

    This also add another way to assess value of the character - voyage reliability or best lowest voyage score
    PlayingSince: 2016-09-16Can we get some more characters from TAS?We finally have Caitians in the game!Character wishlist:
    • Lieutenant M'Ress - got her
    • Amanda Rogers - got her
    • Admiral S'rrel from Star Trek IV: The Voyage Home - not in the game yet
    • Agmar - not in the game yet
    • M'yra - not in the game yet
  • Options
    (HGH)Apollo(HGH)Apollo ✭✭✭✭✭
    edited January 2021
    My farthest voyage without reviving was 11 hrs 16 minutes.

    yaccjooxhw5e.png

    The secondary skill was high and the primary skill was pretty good but the proficiencies of the crew were not high.

    u6fn7kppanli.png

    https://datacore.app/behold?crew=janeway_determined_crew&crew=spock_captain_crew&crew=forrest_admiral_crew&crew=borg_queen_crew&crew=braxton_captain_crew&crew=dahj_activated_crew&crew=dsc_number_one_crew&crew=ruafo_crew&crew=phlox_nanoprobe_crew&crew=janeway_viceadmiral_crew&crew=emh_android_crew&crew=dsc_jett_reno_medic_crew

    Does my voyage fit into your theory? Are there other factors also in play? It seems like with my low proficiencies this voyage should have never gone so long.
    Let’s fly!
  • Options
    Dirk GundersonDirk Gunderson ✭✭✭✭✭
    This theory - particularly as more supporting data is generated - suggests voyage rankings should be recalculated in a fundamentally different manner. Rather than the straight number, the expected variance (difference between high and low proficiency rolls) should be a major part of how we set up the voyage rankings.

    Raw power still counts for something - I would still rather put Locutus on a DIP/SEC voyage over Grilka - but a more realistic ranking of crew performance in voyages will help both the people looking to obtain more consistent voyage lengths and those scraping for every extra minute they can get.

    And rather than trying to build a table of arbitrarily selected criteria (high/medium/low), I would think it worthwhile to try to build a formula that can account for both raw power and the variance. Maybe even one formula for people seeking to reduce the RNG component and another for those seeking to maximize it?
  • Options
    Happy to have been an accidental tester lol.
    Thank you Bylo Band for the very clear analysis.

    As a relatively new player i concentrated on voyages to get SR and chrons asap.
    Behold choices, cites and LTOs needed to improve voyage and if possible serve in multiple game areas. So many were dual. Gauntlet use and voyage. This strategy led to 6 hr voyages within 2 months and 10hr voyages in 5 months. But it also caused about a quarter to fail unless babysitted.
    I used tokens an dilitium to revive in these cases to get to the next dilemma.

    This strategy allowed for a fast buildup of SR and i am sitting on over 50 fully fused SR at lv 1-10. It also allows you to get to long voyages without too many legendaries.

    Now i am slowly replacing high proficiency crew with low proficiency crew, going for guaranteed 10 hrs over longer voyages, even replacing FFFE high proficiency crew with 1* crew sometimes., And voyages do not fail anymore. It is ideal if i am very busy.

    When I have time to babysit, i still go for 11+voyages, you can mix it up .
  • Options
    Bylo BandBylo Band ✭✭✭✭✭
    Do you have a list of the main "high proficiency" crew? This will affect how I prioritize citations.

    To paraphrase Justice Potter Stewart, "I can't define it, but I know it when I see it." When the idea was first hatched this was the obvious first question we tried to answer, and it is not easy in the slightest! The example I slid into the write up with Locutus is a good illustration because we struggled to clearly define his value here, as his maximum proficiency numbers are really high, but his proficiency minimum scores are also pretty good, which limits his range. The two crew that LEAP out as obviously High Proficiency are Defensive Phlox and Kahless, the Unforgettable. Very high maximums, very low minimums.

    My farthest voyage without reviving was 11 hrs 16 minutes.

    Does my voyage fit into your theory? Are there other factors also in play? It seems like with my low proficiencies this voyage should have never gone so long.

    As Lorca pointed out while I was doing the second draft, there is actually a second source of voyage RNG that as far as we know cannot be controlled, and that is the skill selection process for hazard checks. I looked over your crew and it looked slightly on the High Proficiency side but not to any degree I would consider extreme, so you may have shot the moon on your skill checks, or maybe you got a higher than expected percentage of gold/silver skill hazards? In either case, that is an impressive voyage!
  • Options
    Thank you for all the effort put into this! Brilliant work.
  • Options
    ~peregrine~~peregrine~ ✭✭✭✭✭
    Oy ... efficiency, unless you just want glory.

    Sorry, @Banjo1012 . 🖖
    "In the short run, the game defines the players. But in the long run, it's us players who define the game." — Nicky Case, The Evolution of Trust
  • Options
    Ishmael MarxIshmael Marx ✭✭✭✭✭
    Great write up Bylo! I enjoyed the “look behind the curtain” narrative as well as your analysis. In a slide presentation, your bar graph would be the epitome of the “picture is worth a thousand words” concept. Like everyone eventually does, I’ve hit several vastly-exceeded-my-expectations voyages and several how-did-that-fail-to-reach-X-hours voyages. The answer was obviously always RNG, but I’d never really given much thought to where the RNG came from. Mostly I just assumed it was greater or lesser hits on the featured skills, but this will help me think more about the crew factor. Kudos. 🙂
  • Options
    Madman1Madman1 ✭✭✭
    edited January 2021
    Great writeup Bylo and a very interesting and informative read. Thanks for doing this and to everyone involved in gathering the data.
  • Options
    IceCatIceCat ✭✭✭✭✭
    This has been a very interesting read. It, as well as your first part of this project, have given me a lot to think about.

    My only regret was I wasn't an active forum participant when you did Part 1. I would have loved to have helped with that.

    I just went back and read through it all. It was so nice to see an actual discussion of ideas, as opposed to the content we usually get now. I wish there was more of that presently.

    If you do another project like this, please let me know. I'd happily be a part of it. 🖖
  • Options
    IceCatIceCat ✭✭✭✭✭
    Question: I recall a mention of a potential part 3 — is that still happening?
  • Options
    AviTrekAviTrek ✭✭✭✭✭
    What I'd love to see from this(and have wanted for a while) is datacore's estimator given a toggle for what you optimize for. I believe datacore is currently optimizing for average length. But it could just as easily optimize for 25/75 percentile, highest worst length, or highest chance over X hours. If you picked highest chance over 10 hours as your optimization preference, then datacore could select crew taking into account that variability.
  • Options
    Prime LorcaPrime Lorca ✭✭✭✭✭
    AviTrek wrote: »
    What I'd love to see from this(and have wanted for a while) is datacore's estimator given a toggle for what you optimize for. I believe datacore is currently optimizing for average length. But it could just as easily optimize for 25/75 percentile, highest worst length, or highest chance over X hours. If you picked highest chance over 10 hours as your optimization preference, then datacore could select crew taking into account that variability.

    I don't know how Datacore works exactly, but in my efforts to git that 12-hour mark I have realized that you need 2,850+ AM to have even the smallest chance. And even that may be too little AM. That means traits pick a lot of your crew for you when you're going for those long voyages. Basically, I'm saying that the datacore programming would have to me pretty nuanced in order to staff a voyage to get the best chance for 12 hours. What I think would be cool is if datacore could take a voyage that you've staffed manually and give the odds of reaching the 12 hour dilemma. :)
    Farewell 🖖
  • Options
    Thanks for this thread, a very interesting read.

    I do have a question about proficiencies. Do all crew proficiencies get rolled( thus reducing total variance), or does it calculate a total proficiency roll that gets rolled once?

    Suppose you have Locutus (399-783)and Bartender Guinan(358-730) on a voyage as only Dip crew. Do both get rolled or do you get one 757-1513 roll?

    Cheers on your efforts!
  • Options
    ...What I think would be cool is if datacore could take a voyage that you've staffed manually and give the odds of reaching the 12 hour dilemma. :)

    If you have the ranges for each crew and assume a gaussian distribution, it's not terribly hard to calculate a confidence interval for a specific time. The issue I see is the front end data entry. I guess if you have a profile saved, screenshots of both the screen with your skill totals and AM and the one thereafter with all your crew's icons could allow it to pull those. Nobody wants to enter all that in manually.
  • Options
    ExanimusExanimus ✭✭✭✭
    Thanks for this thread, a very interesting read.

    I do have a question about proficiencies. Do all crew proficiencies get rolled( thus reducing total variance), or does it calculate a total proficiency roll that gets rolled once?

    Suppose you have Locutus (399-783)and Bartender Guinan(358-730) on a voyage as only Dip crew. Do both get rolled or do you get one 757-1513 roll?

    Cheers on your efforts!

    Isn't it all the collective? All crew dip bases plus their min prof is collective base with the remaining prof differences as collective range for roll?

    That's one of the factors that seems to complicate labeling a card as high or low prof. One skill could have a very small range while another doesn't. A card with a very wide prof in secondary or tertiary skill could be contributing more than you think to the collective prof range. Aviation Yar for example.
  • Options
    AviTrekAviTrek ✭✭✭✭✭
    AviTrek wrote: »
    What I'd love to see from this(and have wanted for a while) is datacore's estimator given a toggle for what you optimize for. I believe datacore is currently optimizing for average length. But it could just as easily optimize for 25/75 percentile, highest worst length, or highest chance over X hours. If you picked highest chance over 10 hours as your optimization preference, then datacore could select crew taking into account that variability.

    I don't know how Datacore works exactly, but in my efforts to git that 12-hour mark I have realized that you need 2,850+ AM to have even the smallest chance. And even that may be too little AM. That means traits pick a lot of your crew for you when you're going for those long voyages. Basically, I'm saying that the datacore programming would have to me pretty nuanced in order to staff a voyage to get the best chance for 12 hours. What I think would be cool is if datacore could take a voyage that you've staffed manually and give the odds of reaching the 12 hour dilemma. :)

    AM vs crew is all handled by the optimizer. At a basic level what Datacore does is it puts crew in all positions, estimates the length, then tries different crew and sees if the voyage was shorter or longer. If it's shorter it ignores the the option and tries again, if it's longer it saves that new longest and then tries a variation on that to see if it can get even longer. Repeat enough times until it thinks it found the best possible.

    The benefit of this is you don't need to code special logic like look for traits. You just tell dc what value you want to get as high as possible and it keeps trying combinations until it finds the answer.

    The reason why you're seeing the need for high AM is probably because 12 is the limit of what a Voyage can reach, so you need to get lucky on all fronts, high AM, high selection of pri/sec, high rolls towards the end. If any of those miss you won't hit 12. But that doesn't mean am vs skill is any more important going for 12 than 10.
  • Options
    Prime LorcaPrime Lorca ✭✭✭✭✭
    AviTrek wrote: »
    AviTrek wrote: »
    What I'd love to see from this(and have wanted for a while) is datacore's estimator given a toggle for what you optimize for. I believe datacore is currently optimizing for average length. But it could just as easily optimize for 25/75 percentile, highest worst length, or highest chance over X hours. If you picked highest chance over 10 hours as your optimization preference, then datacore could select crew taking into account that variability.

    I don't know how Datacore works exactly, but in my efforts to git that 12-hour mark I have realized that you need 2,850+ AM to have even the smallest chance. And even that may be too little AM. That means traits pick a lot of your crew for you when you're going for those long voyages. Basically, I'm saying that the datacore programming would have to me pretty nuanced in order to staff a voyage to get the best chance for 12 hours. What I think would be cool is if datacore could take a voyage that you've staffed manually and give the odds of reaching the 12 hour dilemma. :)

    AM vs crew is all handled by the optimizer. At a basic level what Datacore does is it puts crew in all positions, estimates the length, then tries different crew and sees if the voyage was shorter or longer. If it's shorter it ignores the the option and tries again, if it's longer it saves that new longest and then tries a variation on that to see if it can get even longer. Repeat enough times until it thinks it found the best possible.

    The benefit of this is you don't need to code special logic like look for traits. You just tell dc what value you want to get as high as possible and it keeps trying combinations until it finds the answer.

    The reason why you're seeing the need for high AM is probably because 12 is the limit of what a Voyage can reach, so you need to get lucky on all fronts, high AM, high selection of pri/sec, high rolls towards the end. If any of those miss you won't hit 12. But that doesn't mean am vs skill is any more important going for 12 than 10.

    That part where you mentioned the good RNG at the end... if you don't have crew with high enough max proficiency, then you won't be able to make those checks at the end regardless of RNG. That's why I mentioned the special staffing thing. There's a difference (though subtle) between maximizing for the longest likely voyage and the most potential to hit 12 hours. That's why I say it would need to be more nuanced than what you described. Though I certainly thank you for the explanation! That was some insight that I needed. :)
    Farewell 🖖
  • Options
    Bylo BandBylo Band ✭✭✭✭✭
    @RaraRacing, you did not miss anything. His numbers place him firmly in the gray area of what would be Medium Proficiency, had we bothered to describe it, lol. For my voyages during this time, I pretty much avoided anybody that wasn't obviously Low Proficiency, so that meant avoiding him. And then without giving any spoilers for the next part, I found another reason to bench Braxton. But good eye :)
  • Options
    Oh my that’s a lot of reading! Right now i’ll just assume how good of a job you did :wink:
    “What's a knockout like you doing in a computer-generated gin joint like this?”

    Proud member of Patterns of Force
    Captain Level 99
    Played since January 2017

    TP: Do better!!!
  • Options
    RaraRacingRaraRacing ✭✭✭✭✭
    Bylo Band wrote: »
    @RaraRacing, you did not miss anything. His numbers place him firmly in the gray area of what would be Medium Proficiency, had we bothered to describe it, lol. For my voyages during this time, I pretty much avoided anybody that wasn't obviously Low Proficiency, so that meant avoiding him. And then without giving any spoilers for the next part, I found another reason to bench Braxton. But good eye :)

    Excellent ... can't wait for part 3 then :)

    Speculation (don't answer this - keep up the suspense) ... Maybe the extra reason is why I like him? ... he has over 20% of his total voyage score in skill #3, which I like because it makes crew flexible to cover various voyage combos (see Borg Queen, RAF O'Brien, Xindi Insectoid, Pollard as some of my fave/most used crew for voyages - FO Burnham also falls into that category, but I only have her 1/5).

    @(HGH)Apollo ... I would think that the following crew in your list do not fall under the "low proficiency" category, while maybe not high, at least solidly in the middle of the pack like Braxton:
    - Captain Spock for CMD
    - Warship EMA (for SEC and CMD)
    (- Braxton (see above), Forrest - ENG)
    But, I could be wrong.

    I'm not sure if the proficiency aspect only counts for the two main voyages skills (if a crew has them) or across all skills.
  • Options
    Prime LorcaPrime Lorca ✭✭✭✭✭
    RaraRacing wrote: »
    I'm not sure if the proficiency aspect only counts for the two main voyages skills (if a crew has them) or across all skills.

    It technically counts for all, but voyages typically go well past the limits of (what I call) tertiary voyage scores. By doing this, it allows the RNG to even out bad rolls with good ones. For that reason, I find that the two main voyage skills provide nearly all of the RNG. Well... all that is within the captain's control, anyway. As Bylo said earlier, the game picks the skills to check throughout the voyage. That is a source of RNG, but not one within the player's control.
    Farewell 🖖
Sign In or Register to comment.