Home The Bridge

Inexplicable Faction Failure Rates

245

Comments

  • I have been informed by someone at DB that the 'Chance of Success' indicators are "...working as intended."

    Note: Not 'as advertised' or 'as posted' but "...as intended."

    😕
  • Gauntlet still heavily favors the defender. In the last 3 days my 96%+ chance to win matches have had losses in the double digits. No matter how many times I tell myself the dice are loaded I still fume when 5%ers go 3/6 and my 25 or 50%er goes 1/6 and I lose by 15 points.
  • The difference 'Regression to the Mean' makes:

    When dealing with analysis of occurrences where the potential n approaches infinity, collecting a sample size large enough to ensure significant results becomes increasingly difficult, often becoming practically impossible.

    Observations of real life data in such situations bares a close resemblance to a standard deviation or bell shaped curve.

    'Regression to the Mean' and 'Deviation from the Norm' serve as tools to measure how closely a data set resembles the expected distribution.

    Such tools are used to make accurate inferences from manageable sample sizes. They allow us to do things like accurately predicting the behavior of large groups from seemingly miniscule sample sizes.

    For instance, using such tools can allow pollsters to predict the outcome of national American elections with a margin of error of 4% from a sample as small as 700 likely voters.

  • For instance, using such tools can allow pollsters to predict the outcome of national American elections with a margin of error of 4% from a sample as small as 700 likely voters.

    Not sure why you would want to use that as an example since every pollster got it wrong in the last election


    SSR Noob
    Spoiler of spoils
  • When DB means a chance of success they mean to say that a 0.001% chance is a chance of success because 1 in a thousand times it could happen. I haven't seen them in a long time but they use to have these "best chance of" packs that meant nothing. You could spend $100 on them and not get the featured card you were supposed to have a "best chance of". Technically they aren't lying if you normally have like a 1 in a thousand chance of getting the card you want and then they make a pack where it's the "best chance" and the stats go to 1 in 950. It's bad business but its technically not dishonest.
  • Not sure why you would want to use that as an example since every pollster got it wrong in the last election

    I used that because I knew someone would say this.

    People misunderstand 'margin of error'

    A 4% margin of error does not mean that the result is going to fall within 4% of the predicted result. It means there is a 90% chance that it will fall in that range.

    Given that Clinton won the popular vote by over 3 million ballots, most national polls for the last US presidential election were correct.

  • When DB means a chance of success they mean to say that a 0.001% chance is a chance of success because 1 in a thousand times it could happen. I haven't seen them in a long time but they use to have these "best chance of" packs that meant nothing. You could spend $100 on them and not get the featured card you were supposed to have a "best chance of". Technically they aren't lying if you normally have like a 1 in a thousand chance of getting the card you want and then they make a pack where it's the "best chance" and the stats go to 1 in 950. It's bad business but its technically not dishonest.

    This is true for the 'best chance' packs, and it was represented accurately in the ℹ tab on those listings to the best of my knowledge. I have only seen 'best chance' packs for ship schematics in the last couple of months.

    There is no such qualification on the 'Chance of Success' gage for shuttle missions however, so the 2 don't directly compare.
  • Nice to see you again Stormy. I'm curious, does your data show similar errors across the range of probable success, or does it get worse the higher your success should be?
    Based on the number of threads on this subject, there seems to be a big error at the 90+% success rates, but I don't read a lot of complaints about the other rates being off.
  • All the ones I tracked have been off.

    The basic data:

    90-99% ~75%

    80-89% ~68%

    70-79% ~61%

    60-69% ~56%

    There are also differences between factions, but that may be a function of the number of crew members per mission.

    I recorded each instance separately, so it's taken time to format the data. I'm also looking at it from a few different angles trying to tease out the details.
  • Boxy McBoxfaceBoxy McBoxface ✭✭✭
    edited March 2018
    Given that Clinton won the popular vote by over 3 million ballots, most national polls for the last US presidential election were correct.

    Just so we're clear, despite everyone picking Clinton to win the presidency and losing, statistically they were correct?

    Here is a prime example of statistics breaking down in the real world because all the data collection in the world won't give you a probable outcome if the initial equation is wrong.

    Yes, all the polls predicted Clinton to win, what they ignored was the fact that winning the popular vote doesn't win you the presidency.

    Maybe you should look at why the displayed chances of success are so out of whack with what you're actually experiencing

    SSR Noob
    Spoiler of spoils
  • Just so we're clear, despite everyone picking Clinton to win the presidency and losing, statistically they were correct?

    You're demonstrating another misunderstanding of the basics of statistical analysis.

    The polls forecast the number of votes for each candidate; analysts in turn use that data to predict winners and losers.

    In the case of the 2018 election the actual vote numbers were within the forecasts' margins of error even though the predictions of the overall results were incorrect.

    This is just one reason why statisticians tend to become bitter over time. 😈

  • Maybe you should look at why the displayed chances of success are so out of whack with what you're actually experiencing

    That is precisely what I'm looking at.

    At the same time, I'm trying to demonstrate the problems that arise from attempting to oversimplifying the problem.

    A Treksplination™; Probability and statistics, like time travel, should give you a headache. If not, you haven't considered all the permutations.
  • RikerWasNumber1RikerWasNumber1 ✭✭✭
    edited March 2018

    Maybe you should look at why the displayed chances of success are so out of whack with what you're actually experiencing

    That is precisely what I'm looking at.

    At the same time, I'm trying to demonstrate the problems that arise from attempting to oversimplifying the problem.

    A Treksplination™; Probability and statistics, like time travel, should give you a headache. If not, you haven't considered all the permutations.

    ~Inappropriate comment snipped[/snipped]
  • <TGE> Clifford<TGE> Clifford ✭✭✭✭✭

    Maybe you should look at why the displayed chances of success are so out of whack with what you're actually experiencing

    That is precisely what I'm looking at.

    At the same time, I'm trying to demonstrate the problems that arise from attempting to oversimplifying the problem.

    A Treksplination™; Probability and statistics, like time travel, should give you a headache. If not, you haven't considered all the permutations.

    I very much agree with you on that last part.

    Sometimes I think about the odds of something, and then I start realizing just how many factors are involved, and so that’s when I decide nope, I don’t need to know.
  • Rowden wrote: »
    If one were to speculate a possible motive is that players are far happier to dil burn or use timers (i.e. spend money) on missions with high displayed success rates.

    or option 3 is that they know it's showing the wrong numbers but can't figure out why

    SSR Noob
    Spoiler of spoils
  • PenguinJimPenguinJim ✭✭✭✭✭
    Everyone seems to assume "Chance of Success" literally refers to how the game server calculates the odds of the shuttle's success. For all we know, it doesn't. It could be that faction's estimate of your shuttle's chance of success, for example. After all, the game doesn't literally power up a warp core when it loads. It may well be showing wrong numbers for shuttle success - intentionally.

    Personally, I treat Timelines as a game (shocking, I know!). The numbers it gives me don't necessarily reflect how effective a combination will be in practice. As in other games, like Diablo III or The Division, the numbers are a good starting point, and I use my own experience to build on those numbers. (I now use tactics for shuttle missions that produce results above their predicted "Chance of Success", so it can be done.)
  • ...
    PenguinJim wrote: »
    Everyone seems to assume "Chance of Success" literally refers to how the game server calculates the odds of the shuttle's success. For all we know, it doesn't. It could be that faction's estimate of your shuttle's chance of success, for example. After all, the game doesn't literally power up a warp core when it loads. It may well be showing wrong numbers for shuttle success - intentionally.

    Personally, I treat Timelines as a game (shocking, I know!). The numbers it gives me don't necessarily reflect how effective a combination will be in practice. As in other games, like Diablo III or The Division, the numbers are a good starting point, and I use my own experience to build on those numbers. (I now use tactics for shuttle missions that produce results above their predicted "Chance of Success", so it can be done.)

    The reason why I started tracking non event missions [which are the ones Rowden analyzed] was because Alex aka Nod [a former DB employee, for those that do not know] insisted on the forums that the displayed shuttle percentages were correct, after running his test using a large number (don't remember how many) of missions. Once I reached around 1000 + missions, the displayed and actual percentages did come very close to each other. At one time, the values were within less than a percent of each other over the total missions. But that is not the same when you look at the more recent data for the same amount of missions.
  • There are some of us trying to log our shuttle display vs. actual success rate. More data is always welcome.

    https://goo.gl/forms/POtXyD6EJhc7yxmy1

    PM me if you have a large amount of data to contribute.
  • Here's a link to a post on the old forums from Nod on 10/1/2017, quite a while after I started tracking my missions: https://forums.disruptorbeam.com/stt/viewreply/376508/

    The post is in response to an initial post by [RO]BitterDevil
    Here's the text from BitterDevil's post:
    I just had a long and frustrating discussion with PS and they eventually confirmed that it doesn’t matter the percentage shown, the game is designed so that you must fail.

    Good to know!

    Here’s a thought…since this is true for the shuttles…what does it say for the rest of the game?

    And here's Nod's response:
    To be clear, shuttle percentages are not manipulated by us as events go on, and they have been thoroughly tested by us over time and confirmed to be accurate (within a reasonable margin of error). I believe several player initiatives have shown that, when combined across players to reach a significant sample size, the percentage success chances do match what is displayed.

    An 80% chance of success does not mean that your shuttles will always win four times in a row and fail the fifth attempt; it means that, over time, you will succeed 80% of the time. That doesn’t mean that streaks of bad or good luck won’t occur, but it does mean that, over time and with a statistically valid sample size, they even out to be correct.

    You can consider my post to be an authoritative post that we don’t rig the odds, and that the odds are calculated by the aggregate skill score of your core skills, plus other bonuses, compared against the difficulty of the current shuttle tier you’re up against. We don’t flip those odds, intentionally misrepresent them, or change them over time. If you believe there is a bug present, please submit a bug to support and they’ll pass it along to the team.

    Nod
    Product Manager, Star Trek Timelines

    Are 2000 shuttles considered enough of a "statistically valid sample size" ?? If so, then Nod's statement above about accuracy is no longer valid.
  • Some excellent information added, thank you.

    As I mentioned earlier, I was told by someone with DB that the Chance of Success was working 'as intended'.

    This response will be familiar to anyone who has questioned DB about their RNG's.

    When I brought up that this in no way addressed whether it was working as stated or as advertised I received a response which completely misstated the nature of a 'weighted roll' and did not attempt to address my concern. It was, in short, a shameless attempt at gas-lighting.

    I can only conclude that DB is deliberately posting inaccurate information and has no intention of changing that.
  • milamber42 wrote: »

    Are 2000 shuttles considered enough of a "statistically valid sample size" ?? If so, then Nod's statement above about accuracy is no longer valid.

    2,000 instances is statistically significant as long as the right analysis is applied.
  • Commander SinclairCommander Sinclair ✭✭✭✭✭
    Is it just me, or do the failures occur at approximately the same time every day? I have been running consistent shuttles during the morning and afternoon with the same factions for over a week, and the same faction shuttles fail every day for about the same 5-6 hour range (6am-12pm EST).

    Anyone else run into this?
    I want to become a Dilionaire...
  • Is it just me, or do the failures occur at approximately the same time every day? I have been running consistent shuttles during the morning and afternoon with the same factions for over a week, and the same faction shuttles fail every day for about the same 5-6 hour range (6am-12pm EST).

    Anyone else run into this?

    I'm not sure I understand.

    Are you saying that any shuttle mission from a cdrtain faction is more likely to fail at certain times of day?
    i.e. any Federation shuttle is more likely to fail between 9am & noon than between 9pm & midnight?
  • Commander SinclairCommander Sinclair ✭✭✭✭✭
    Yes, exactly. I generally run 3 hour shuttles right before I turn in for the night, usually 2x Augments and 2x Sec31. Most days they are 100% success.

    When I clear those in the morning, I do 1x Sec31, 1xAugment, and 2x Fed. Generally ALL fail. 3 hours later, usually I get 1 success and 3 fails. EVERY...SINGLE...DAY

    After lunch, it calms down and I generally get 100% successes throughout the rest of the day, with occasional fails. (Maybe 1 in 20 or so)

    Now, maybe this is due to the successes from the night shuttles making the morning ones harder, but the percentages and sometimes even the missions are the same. (75-85%) The afternoon percentages are MUCH lower, average 58-65% & succeed more often.

    I am tempted to keep a formal log, which I have done several times in the past, but I just haven't had the time in the past month or so.
    I want to become a Dilionaire...
  • NeemsNeems ✭✭✭

    As I mentioned earlier, I was told by someone with DB that the Chance of Success was working 'as intended'.

    This isn't meant to be a judgment on anything else you're saying, but that phrase may not be as dark as you're thinking. I work in IT and dabble in customer service, and "working as intended" is standard lingo for "it works".
  • Neems wrote: »

    As I mentioned earlier, I was told by someone with DB that the Chance of Success was working 'as intended'.

    This isn't meant to be a judgment on anything else you're saying, but that phrase may not be as dark as you're thinking. I work in IT and dabble in customer service, and "working as intended" is standard lingo for "it works".

    I am aware of that.

    However when I have brought up that the statement is potentially misleading, I have never received a response offering the requested clarification.

    Instead I've received completely unrelated explanations, most recently a thoroughly inaccurate explanation of 'weighted rolls'.

    If DB is in fact not attempting to lie to and gas-light their customers then they need to seriously address their methods of communication, as an impartial analysis suggests that deception is the only reasonable conclusion to be drawn.
  • Yes, exactly. I generally run 3 hour shuttles right before I turn in for the night, usually 2x Augments and 2x Sec31. Most days they are 100% success.

    When I clear those in the morning, I do 1x Sec31, 1xAugment, and 2x Fed. Generally ALL fail. 3 hours later, usually I get 1 success and 3 fails. EVERY...SINGLE...DAY

    After lunch, it calms down and I generally get 100% successes throughout the rest of the day, with occasional fails. (Maybe 1 in 20 or so)

    Now, maybe this is due to the successes from the night shuttles making the morning ones harder, but the percentages and sometimes even the missions are the same. (75-85%) The afternoon percentages are MUCH lower, average 58-65% & succeed more often.

    I am tempted to keep a formal log, which I have done several times in the past, but I just haven't had the time in the past month or so.

    I have not noticed anything similar, however I send different shuttles from various factions in no particular order and at different times. I also haven't tracked the times I send shuttles, so my records are of no help.

    My gut feeling, for what it's worth, is that if there is a correlation it is an unintentional side-effect of other biases in the RNG matrix. It would require a lot of coding to create the described results, and I can't think of any benefit to DB resulting from it.

    I would strongly encourage you to track your results. Because of the specificities of the situation you describe it won't require thousands of data points to arrive at statistically significant results. A few hundred should be sufficient, at least to suggest whether a larger sample is warranted.
  • Commander SinclairCommander Sinclair ✭✭✭✭✭
    biases in the RNG matrix.

    I LOVE this comment - it sums up everything wrong with this game. By definition, the RNG cannot have bias, yet seems to often glitch on certain things.
    I want to become a Dilionaire...
Sign In or Register to comment.