Gauntlet still heavily favors the defender. In the last 3 days my 96%+ chance to win matches have had losses in the double digits. No matter how many times I tell myself the dice are loaded I still fume when 5%ers go 3/6 and my 25 or 50%er goes 1/6 and I lose by 15 points.
When dealing with analysis of occurrences where the potential n approaches infinity, collecting a sample size large enough to ensure significant results becomes increasingly difficult, often becoming practically impossible.
Observations of real life data in such situations bares a close resemblance to a standard deviation or bell shaped curve.
'Regression to the Mean' and 'Deviation from the Norm' serve as tools to measure how closely a data set resembles the expected distribution.
Such tools are used to make accurate inferences from manageable sample sizes. They allow us to do things like accurately predicting the behavior of large groups from seemingly miniscule sample sizes.
For instance, using such tools can allow pollsters to predict the outcome of national American elections with a margin of error of 4% from a sample as small as 700 likely voters.
For instance, using such tools can allow pollsters to predict the outcome of national American elections with a margin of error of 4% from a sample as small as 700 likely voters.
Not sure why you would want to use that as an example since every pollster got it wrong in the last election
When DB means a chance of success they mean to say that a 0.001% chance is a chance of success because 1 in a thousand times it could happen. I haven't seen them in a long time but they use to have these "best chance of" packs that meant nothing. You could spend $100 on them and not get the featured card you were supposed to have a "best chance of". Technically they aren't lying if you normally have like a 1 in a thousand chance of getting the card you want and then they make a pack where it's the "best chance" and the stats go to 1 in 950. It's bad business but its technically not dishonest.
Not sure why you would want to use that as an example since every pollster got it wrong in the last election
I used that because I knew someone would say this.
People misunderstand 'margin of error'
A 4% margin of error does not mean that the result is going to fall within 4% of the predicted result. It means there is a 90% chance that it will fall in that range.
Given that Clinton won the popular vote by over 3 million ballots, most national polls for the last US presidential election were correct.
When DB means a chance of success they mean to say that a 0.001% chance is a chance of success because 1 in a thousand times it could happen. I haven't seen them in a long time but they use to have these "best chance of" packs that meant nothing. You could spend $100 on them and not get the featured card you were supposed to have a "best chance of". Technically they aren't lying if you normally have like a 1 in a thousand chance of getting the card you want and then they make a pack where it's the "best chance" and the stats go to 1 in 950. It's bad business but its technically not dishonest.
This is true for the 'best chance' packs, and it was represented accurately in the ℹ tab on those listings to the best of my knowledge. I have only seen 'best chance' packs for ship schematics in the last couple of months.
There is no such qualification on the 'Chance of Success' gage for shuttle missions however, so the 2 don't directly compare.
Nice to see you again Stormy. I'm curious, does your data show similar errors across the range of probable success, or does it get worse the higher your success should be?
Based on the number of threads on this subject, there seems to be a big error at the 90+% success rates, but I don't read a lot of complaints about the other rates being off.
There are also differences between factions, but that may be a function of the number of crew members per mission.
I recorded each instance separately, so it's taken time to format the data. I'm also looking at it from a few different angles trying to tease out the details.
Given that Clinton won the popular vote by over 3 million ballots, most national polls for the last US presidential election were correct.
Just so we're clear, despite everyone picking Clinton to win the presidency and losing, statistically they were correct?
Here is a prime example of statistics breaking down in the real world because all the data collection in the world won't give you a probable outcome if the initial equation is wrong.
Yes, all the polls predicted Clinton to win, what they ignored was the fact that winning the popular vote doesn't win you the presidency.
Maybe you should look at why the displayed chances of success are so out of whack with what you're actually experiencing
Just so we're clear, despite everyone picking Clinton to win the presidency and losing, statistically they were correct?
You're demonstrating another misunderstanding of the basics of statistical analysis.
The polls forecast the number of votes for each candidate; analysts in turn use that data to predict winners and losers.
In the case of the 2018 election the actual vote numbers were within the forecasts' margins of error even though the predictions of the overall results were incorrect.
This is just one reason why statisticians tend to become bitter over time. 😈
Maybe you should look at why the displayed chances of success are so out of whack with what you're actually experiencing
That is precisely what I'm looking at.
At the same time, I'm trying to demonstrate the problems that arise from attempting to oversimplifying the problem.
A Treksplination™; Probability and statistics, like time travel, should give you a headache. If not, you haven't considered all the permutations.
I very much agree with you on that last part.
Sometimes I think about the odds of something, and then I start realizing just how many factors are involved, and so that’s when I decide nope, I don’t need to know.
You are correct that the odds do not average out currently.
I have analysed a dataset containing 4,000 missions, which were all done outside events and no shuttle boosts were used. I split the analysis into two sections, data gathered from shuttles sent before April 2017, and data gathered between April 2017 and November 2017. Both populations have a size of around 2,000 missions each.
Interestingly, for shuttles sent before April 2017 the predicted success is very similar to the actual, however, since April the success rate of shuttles displayed as 60-69%, 70-79%, 80-89% and 90-100% are all below. Therefore, the data clearly indicates that currently players do indeed get a worse shuttle success rate for shuttles sent in the 60-100% bracket.
I sent this analysis to DB last November and offered to share the dataset with them. Quite frankly they did not look at the graphs I supplied them or ask for the dataset, or ask any questions regarding the data or the analysis. No interest in a dataset of around 4,000 missions over several months leads me to believe that they are very aware that the displayed shuttle success rate is not accurate, but have no intention to fix it. If one were to speculate a possible motive is that players are far happier to dil burn or use timers (i.e. spend money) on missions with high displayed success rates.
If one were to speculate a possible motive is that players are far happier to dil burn or use timers (i.e. spend money) on missions with high displayed success rates.
or option 3 is that they know it's showing the wrong numbers but can't figure out why
Everyone seems to assume "Chance of Success" literally refers to how the game server calculates the odds of the shuttle's success. For all we know, it doesn't. It could be that faction's estimate of your shuttle's chance of success, for example. After all, the game doesn't literally power up a warp core when it loads. It may well be showing wrong numbers for shuttle success - intentionally.
Personally, I treat Timelines as a game (shocking, I know!). The numbers it gives me don't necessarily reflect how effective a combination will be in practice. As in other games, like Diablo III or The Division, the numbers are a good starting point, and I use my own experience to build on those numbers. (I now use tactics for shuttle missions that produce results above their predicted "Chance of Success", so it can be done.)
Everyone seems to assume "Chance of Success" literally refers to how the game server calculates the odds of the shuttle's success. For all we know, it doesn't. It could be that faction's estimate of your shuttle's chance of success, for example. After all, the game doesn't literally power up a warp core when it loads. It may well be showing wrong numbers for shuttle success - intentionally.
Personally, I treat Timelines as a game (shocking, I know!). The numbers it gives me don't necessarily reflect how effective a combination will be in practice. As in other games, like Diablo III or The Division, the numbers are a good starting point, and I use my own experience to build on those numbers. (I now use tactics for shuttle missions that produce results above their predicted "Chance of Success", so it can be done.)
The reason why I started tracking non event missions [which are the ones Rowden analyzed] was because Alex aka Nod [a former DB employee, for those that do not know] insisted on the forums that the displayed shuttle percentages were correct, after running his test using a large number (don't remember how many) of missions. Once I reached around 1000 + missions, the displayed and actual percentages did come very close to each other. At one time, the values were within less than a percent of each other over the total missions. But that is not the same when you look at the more recent data for the same amount of missions.
The post is in response to an initial post by [RO]BitterDevil
Here's the text from BitterDevil's post:
I just had a long and frustrating discussion with PS and they eventually confirmed that it doesn’t matter the percentage shown, the game is designed so that you must fail.
Good to know!
Here’s a thought…since this is true for the shuttles…what does it say for the rest of the game?
And here's Nod's response:
To be clear, shuttle percentages are not manipulated by us as events go on, and they have been thoroughly tested by us over time and confirmed to be accurate (within a reasonable margin of error). I believe several player initiatives have shown that, when combined across players to reach a significant sample size, the percentage success chances do match what is displayed.
An 80% chance of success does not mean that your shuttles will always win four times in a row and fail the fifth attempt; it means that, over time, you will succeed 80% of the time. That doesn’t mean that streaks of bad or good luck won’t occur, but it does mean that, over time and with a statistically valid sample size, they even out to be correct.
You can consider my post to be an authoritative post that we don’t rig the odds, and that the odds are calculated by the aggregate skill score of your core skills, plus other bonuses, compared against the difficulty of the current shuttle tier you’re up against. We don’t flip those odds, intentionally misrepresent them, or change them over time. If you believe there is a bug present, please submit a bug to support and they’ll pass it along to the team.
Nod
Product Manager, Star Trek Timelines
Are 2000 shuttles considered enough of a "statistically valid sample size" ?? If so, then Nod's statement above about accuracy is no longer valid.
As I mentioned earlier, I was told by someone with DB that the Chance of Success was working 'as intended'.
This response will be familiar to anyone who has questioned DB about their RNG's.
When I brought up that this in no way addressed whether it was working as stated or as advertised I received a response which completely misstated the nature of a 'weighted roll' and did not attempt to address my concern. It was, in short, a shameless attempt at gas-lighting.
I can only conclude that DB is deliberately posting inaccurate information and has no intention of changing that.
Is it just me, or do the failures occur at approximately the same time every day? I have been running consistent shuttles during the morning and afternoon with the same factions for over a week, and the same faction shuttles fail every day for about the same 5-6 hour range (6am-12pm EST).
Is it just me, or do the failures occur at approximately the same time every day? I have been running consistent shuttles during the morning and afternoon with the same factions for over a week, and the same faction shuttles fail every day for about the same 5-6 hour range (6am-12pm EST).
Anyone else run into this?
I'm not sure I understand.
Are you saying that any shuttle mission from a cdrtain faction is more likely to fail at certain times of day?
i.e. any Federation shuttle is more likely to fail between 9am & noon than between 9pm & midnight?
Yes, exactly. I generally run 3 hour shuttles right before I turn in for the night, usually 2x Augments and 2x Sec31. Most days they are 100% success.
When I clear those in the morning, I do 1x Sec31, 1xAugment, and 2x Fed. Generally ALL fail. 3 hours later, usually I get 1 success and 3 fails. EVERY...SINGLE...DAY
After lunch, it calms down and I generally get 100% successes throughout the rest of the day, with occasional fails. (Maybe 1 in 20 or so)
Now, maybe this is due to the successes from the night shuttles making the morning ones harder, but the percentages and sometimes even the missions are the same. (75-85%) The afternoon percentages are MUCH lower, average 58-65% & succeed more often.
I am tempted to keep a formal log, which I have done several times in the past, but I just haven't had the time in the past month or so.
As I mentioned earlier, I was told by someone with DB that the Chance of Success was working 'as intended'.
This isn't meant to be a judgment on anything else you're saying, but that phrase may not be as dark as you're thinking. I work in IT and dabble in customer service, and "working as intended" is standard lingo for "it works".
As I mentioned earlier, I was told by someone with DB that the Chance of Success was working 'as intended'.
This isn't meant to be a judgment on anything else you're saying, but that phrase may not be as dark as you're thinking. I work in IT and dabble in customer service, and "working as intended" is standard lingo for "it works".
I am aware of that.
However when I have brought up that the statement is potentially misleading, I have never received a response offering the requested clarification.
Instead I've received completely unrelated explanations, most recently a thoroughly inaccurate explanation of 'weighted rolls'.
If DB is in fact not attempting to lie to and gas-light their customers then they need to seriously address their methods of communication, as an impartial analysis suggests that deception is the only reasonable conclusion to be drawn.
Yes, exactly. I generally run 3 hour shuttles right before I turn in for the night, usually 2x Augments and 2x Sec31. Most days they are 100% success.
When I clear those in the morning, I do 1x Sec31, 1xAugment, and 2x Fed. Generally ALL fail. 3 hours later, usually I get 1 success and 3 fails. EVERY...SINGLE...DAY
After lunch, it calms down and I generally get 100% successes throughout the rest of the day, with occasional fails. (Maybe 1 in 20 or so)
Now, maybe this is due to the successes from the night shuttles making the morning ones harder, but the percentages and sometimes even the missions are the same. (75-85%) The afternoon percentages are MUCH lower, average 58-65% & succeed more often.
I am tempted to keep a formal log, which I have done several times in the past, but I just haven't had the time in the past month or so.
I have not noticed anything similar, however I send different shuttles from various factions in no particular order and at different times. I also haven't tracked the times I send shuttles, so my records are of no help.
My gut feeling, for what it's worth, is that if there is a correlation it is an unintentional side-effect of other biases in the RNG matrix. It would require a lot of coding to create the described results, and I can't think of any benefit to DB resulting from it.
I would strongly encourage you to track your results. Because of the specificities of the situation you describe it won't require thousands of data points to arrive at statistically significant results. A few hundred should be sufficient, at least to suggest whether a larger sample is warranted.
I LOVE this comment - it sums up everything wrong with this game. By definition, the RNG cannot have bias, yet seems to often glitch on certain things.
Comments
Note: Not 'as advertised' or 'as posted' but "...as intended."
😕
When dealing with analysis of occurrences where the potential n approaches infinity, collecting a sample size large enough to ensure significant results becomes increasingly difficult, often becoming practically impossible.
Observations of real life data in such situations bares a close resemblance to a standard deviation or bell shaped curve.
'Regression to the Mean' and 'Deviation from the Norm' serve as tools to measure how closely a data set resembles the expected distribution.
Such tools are used to make accurate inferences from manageable sample sizes. They allow us to do things like accurately predicting the behavior of large groups from seemingly miniscule sample sizes.
For instance, using such tools can allow pollsters to predict the outcome of national American elections with a margin of error of 4% from a sample as small as 700 likely voters.
Not sure why you would want to use that as an example since every pollster got it wrong in the last election
SSR Noob
Spoiler of spoils
I used that because I knew someone would say this.
People misunderstand 'margin of error'
A 4% margin of error does not mean that the result is going to fall within 4% of the predicted result. It means there is a 90% chance that it will fall in that range.
Given that Clinton won the popular vote by over 3 million ballots, most national polls for the last US presidential election were correct.
This is true for the 'best chance' packs, and it was represented accurately in the ℹ tab on those listings to the best of my knowledge. I have only seen 'best chance' packs for ship schematics in the last couple of months.
There is no such qualification on the 'Chance of Success' gage for shuttle missions however, so the 2 don't directly compare.
Based on the number of threads on this subject, there seems to be a big error at the 90+% success rates, but I don't read a lot of complaints about the other rates being off.
The basic data:
90-99% ~75%
80-89% ~68%
70-79% ~61%
60-69% ~56%
There are also differences between factions, but that may be a function of the number of crew members per mission.
I recorded each instance separately, so it's taken time to format the data. I'm also looking at it from a few different angles trying to tease out the details.
Just so we're clear, despite everyone picking Clinton to win the presidency and losing, statistically they were correct?
Here is a prime example of statistics breaking down in the real world because all the data collection in the world won't give you a probable outcome if the initial equation is wrong.
Yes, all the polls predicted Clinton to win, what they ignored was the fact that winning the popular vote doesn't win you the presidency.
Maybe you should look at why the displayed chances of success are so out of whack with what you're actually experiencing
SSR Noob
Spoiler of spoils
You're demonstrating another misunderstanding of the basics of statistical analysis.
The polls forecast the number of votes for each candidate; analysts in turn use that data to predict winners and losers.
In the case of the 2018 election the actual vote numbers were within the forecasts' margins of error even though the predictions of the overall results were incorrect.
This is just one reason why statisticians tend to become bitter over time. 😈
That is precisely what I'm looking at.
At the same time, I'm trying to demonstrate the problems that arise from attempting to oversimplifying the problem.
A Treksplination™; Probability and statistics, like time travel, should give you a headache. If not, you haven't considered all the permutations.
~Inappropriate comment snipped[/snipped]
I very much agree with you on that last part.
Sometimes I think about the odds of something, and then I start realizing just how many factors are involved, and so that’s when I decide nope, I don’t need to know.
I have analysed a dataset containing 4,000 missions, which were all done outside events and no shuttle boosts were used. I split the analysis into two sections, data gathered from shuttles sent before April 2017, and data gathered between April 2017 and November 2017. Both populations have a size of around 2,000 missions each.
Interestingly, for shuttles sent before April 2017 the predicted success is very similar to the actual, however, since April the success rate of shuttles displayed as 60-69%, 70-79%, 80-89% and 90-100% are all below. Therefore, the data clearly indicates that currently players do indeed get a worse shuttle success rate for shuttles sent in the 60-100% bracket.
I sent this analysis to DB last November and offered to share the dataset with them. Quite frankly they did not look at the graphs I supplied them or ask for the dataset, or ask any questions regarding the data or the analysis. No interest in a dataset of around 4,000 missions over several months leads me to believe that they are very aware that the displayed shuttle success rate is not accurate, but have no intention to fix it. If one were to speculate a possible motive is that players are far happier to dil burn or use timers (i.e. spend money) on missions with high displayed success rates.
or option 3 is that they know it's showing the wrong numbers but can't figure out why
SSR Noob
Spoiler of spoils
Personally, I treat Timelines as a game (shocking, I know!). The numbers it gives me don't necessarily reflect how effective a combination will be in practice. As in other games, like Diablo III or The Division, the numbers are a good starting point, and I use my own experience to build on those numbers. (I now use tactics for shuttle missions that produce results above their predicted "Chance of Success", so it can be done.)
The reason why I started tracking non event missions [which are the ones Rowden analyzed] was because Alex aka Nod [a former DB employee, for those that do not know] insisted on the forums that the displayed shuttle percentages were correct, after running his test using a large number (don't remember how many) of missions. Once I reached around 1000 + missions, the displayed and actual percentages did come very close to each other. At one time, the values were within less than a percent of each other over the total missions. But that is not the same when you look at the more recent data for the same amount of missions.
https://goo.gl/forms/POtXyD6EJhc7yxmy1
PM me if you have a large amount of data to contribute.
The post is in response to an initial post by [RO]BitterDevil
Here's the text from BitterDevil's post:
And here's Nod's response:
Are 2000 shuttles considered enough of a "statistically valid sample size" ?? If so, then Nod's statement above about accuracy is no longer valid.
As I mentioned earlier, I was told by someone with DB that the Chance of Success was working 'as intended'.
This response will be familiar to anyone who has questioned DB about their RNG's.
When I brought up that this in no way addressed whether it was working as stated or as advertised I received a response which completely misstated the nature of a 'weighted roll' and did not attempt to address my concern. It was, in short, a shameless attempt at gas-lighting.
I can only conclude that DB is deliberately posting inaccurate information and has no intention of changing that.
2,000 instances is statistically significant as long as the right analysis is applied.
Anyone else run into this?
I'm not sure I understand.
Are you saying that any shuttle mission from a cdrtain faction is more likely to fail at certain times of day?
i.e. any Federation shuttle is more likely to fail between 9am & noon than between 9pm & midnight?
When I clear those in the morning, I do 1x Sec31, 1xAugment, and 2x Fed. Generally ALL fail. 3 hours later, usually I get 1 success and 3 fails. EVERY...SINGLE...DAY
After lunch, it calms down and I generally get 100% successes throughout the rest of the day, with occasional fails. (Maybe 1 in 20 or so)
Now, maybe this is due to the successes from the night shuttles making the morning ones harder, but the percentages and sometimes even the missions are the same. (75-85%) The afternoon percentages are MUCH lower, average 58-65% & succeed more often.
I am tempted to keep a formal log, which I have done several times in the past, but I just haven't had the time in the past month or so.
This isn't meant to be a judgment on anything else you're saying, but that phrase may not be as dark as you're thinking. I work in IT and dabble in customer service, and "working as intended" is standard lingo for "it works".
I am aware of that.
However when I have brought up that the statement is potentially misleading, I have never received a response offering the requested clarification.
Instead I've received completely unrelated explanations, most recently a thoroughly inaccurate explanation of 'weighted rolls'.
If DB is in fact not attempting to lie to and gas-light their customers then they need to seriously address their methods of communication, as an impartial analysis suggests that deception is the only reasonable conclusion to be drawn.
I have not noticed anything similar, however I send different shuttles from various factions in no particular order and at different times. I also haven't tracked the times I send shuttles, so my records are of no help.
My gut feeling, for what it's worth, is that if there is a correlation it is an unintentional side-effect of other biases in the RNG matrix. It would require a lot of coding to create the described results, and I can't think of any benefit to DB resulting from it.
I would strongly encourage you to track your results. Because of the specificities of the situation you describe it won't require thousands of data points to arrive at statistically significant results. A few hundred should be sufficient, at least to suggest whether a larger sample is warranted.
I LOVE this comment - it sums up everything wrong with this game. By definition, the RNG cannot have bias, yet seems to often glitch on certain things.