Forum Settings
       
This Forum is Read Only

+Craftmanship gear effect on craftingFollow

#1 Oct 26 2010 at 9:08 PM Rating: Good
*
216 posts
Given the amount of frustration surrounding the randomness of crafting, I was curious about what exactly +Craftmanship gear does for you while crafting, so I did an unscientific test to see if anything could be identified.

I took a Rank18 Leatherworker and attempted to synthesize 5 Dodo Leathers first in some low crafting gear, and then again with no gear whatsoever. In both cases I used the Common Tannery support.

The equipment worn for the test was rather low gear that granted a total of +18 Craftmanship +8Mag.Craftmanship, and +13 Control. This is not a huge amount, but with +1 Craftmanship for each rank, we should be able to get some sense of what it does.

At no time during either test did I use Rapid, Bold, or Wait. I just spammed Standard all the way through, regardless of orb color.

The abilities equipped were Fulfillment and Maker's Muse. In both "tests", I used the abilities as I normally would. I've not looked at the information specific to while the abilities were active. In future tests I'd like to try synthing without using the abilities, to get more accurate results.

In the following post I will paste the results of the 5 synthesis while wearing the +craftmanship gear.
In the post following that one, I will paste the results of the 5 synthesis while wearing no gear.
At the bottom of each of the posts, there is an overall average using the information from all 5 synths.

If anyone finds any errors in my numbers let me know. Math and little sleep can be unpredictable.

Results: wrote:
I'll put this before the data.

While a mere 5 synths are too few to be able to draw any firm conclusions from, it does seem to appear that there are no dramatic differences identifiable between synthing with no gear versus synthing with low gear.

With the action outcomes and average Progress so close between the two tests, it seems that if you were going to fail a synth while wearing no gear, wearing gear wouldn't have done a whole lot to avoid this.

This is with relatively low gear though. This needs to be tested again with a little higher gear, so that we can see the gradual increase in success rates.

I would like to invite other crafters to contribute logs of their crafting using gear versus no gear. You can do this by completing the crafting, then highlighting your chat log ingame, pressing ctrl+c, then pasting (ctrl+v) the clipboard into a program like Wordpad. Include the attributes your gear gives, what you were attempting to make, any buffs/abilities you are using, and your rank/class. If you don't want to find the averages yourself, PM the log to me and I'll do it then post it.

It's about time we figure out just how to tackle the crafting randomness, if it is at all possible.


Edited, Oct 26th 2010 11:18pm by theweenie
#2 Oct 26 2010 at 9:08 PM Rating: Excellent
*
216 posts
You use Standard Synthesis.
Progress increases by 14%.
Durability decreases by 10.
Quality increases by 2.
You use Standard Synthesis. The attempt fails!
Progress increases by 3%.
Durability decreases by 21.
You use Standard Synthesis.
Progress increases by 11%.
Durability decreases by 10.
Quality increases by 2.
You use Maker's Muse.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 14%.
Durability decreases by 6.
Quality increases by 6.
You use Standard Synthesis.
Progress increases by 13%.
Durability decreases by 11.
Quality increases by 2.
You use Fulfillment.
Your Maker's Muse effect fades.
You use Standard Synthesis. The attempt fails!
Progress increases by 7%.
Durability decreases by 16.
You use Standard Synthesis. The attempt fails!
Progress increases by 3%.
Durability decreases by 20.
You use Standard Synthesis. The attempt fails!
Progress increases by 4%.
Durability decreases by 22.
You botch the synthesis!

Total:
69% Progress ; 116 Durability loss
1 Successful action, 4 Failures, 3 Neutral
Averages:
14% Progress ; 6 Durability loss on Successful actions
3.5% Progress ; 19.75 Durability loss on Failures
12.7% Progress ; 10.3 Durability loss on Neutral outcomes

You use Standard Synthesis.
Progress increases by 11%.
Durability decreases by 9.
Quality increases by 3.
You use Standard Synthesis. The attempt fails!
Progress increases by 3%.
Durability decreases by 19.
You use Fulfillment.
You use Standard Synthesis.
Progress increases by 13%.
Durability decreases by 11.
Quality increases by 2.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 14%.
Durability decreases by 5.
Quality increases by 4.
You use Standard Synthesis. The attempt fails!
Progress increases by 6%.
Durability decreases by 15.
You use Standard Synthesis.
Progress increases by 13%.
Durability decreases by 9.
Quality increases by 2.
Your Fulfillment effect fades.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 14%.
Durability decreases by 5.
Quality increases by 4.
You use Maker's Muse.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 14%.
Durability decreases by 6.
Quality increases by 7.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 11%.
Quality increases by 10.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 13%.
Durability decreases by 5.
Quality increases by 5.
You create 4 circles of dodo leather.

Total:
112% Progress ; 84 Durability loss
5 Successful action, 2 Failures, 3 Neutral
Averages:
13.2% Progress ; 4.2 Durability loss on Successful actions
4.5% Progress ; 17 Durability loss on Failures
12.3% Progress ; 9.6 Durability loss on Neutral outcomes

You use Standard Synthesis. The attempt succeeds!
Progress increases by 12%.
Quality increases by 8.
You use Fulfillment.
You use Standard Synthesis. The attempt fails!
Progress increases by 3%.
Durability decreases by 21.
You use Standard Synthesis. The attempt fails!
Progress increases by 6%.
Durability decreases by 15.
You use Standard Synthesis.
Progress increases by 11%.
Durability decreases by 9.
Quality increases by 2.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 13%.
Quality increases by 8.
Your Fulfillment effect fades.
You use Maker's Muse.
You use Standard Synthesis. The attempt fails!
Progress increases by 3%.
Durability decreases by 20.
You use Standard Synthesis. The attempt fails!
Progress increases by 8%.
Durability decreases by 14.
You use Standard Synthesis.
Progress increases by 12%.
Durability decreases by 10.
Quality increases by 3.
You use Standard Synthesis.
Progress increases by 13%.
Durability decreases by 11.
Quality increases by 2.
Your Maker's Muse effect fades.
You use Standard Synthesis.
Progress increases by 13%.
Durability decreases by 11.
Quality increases by 3.
You botch the synthesis!

Total:
94% Progress ; 111 Durability loss
2 Successful action, 4 Failures, 4 Neutral
Averages:
12.5% Progress ; 0 Durability loss on Successful actions
5% Progress ; 17.5 Durability loss on Failures
12.3% Progress ; 10.3 Durability loss on Neutral outcomes

You use Maker's Muse.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 12%.
Quality increases by 7.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 14%.
Quality increases by 7.
You use Standard Synthesis.
Progress increases by 12%.
Durability decreases by 10.
Quality increases by 2.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 15%.
Durability decreases by 5.
Quality increases by 6.
Your Maker's Muse effect fades.
You use Fulfillment.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 14%.
Quality increases by 11.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 11%.
Durability decreases by 5.
Quality increases by 7.
You use Standard Synthesis. The attempt fails!
Progress increases by 7%.
Durability decreases by 17.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 16%.
Durability decreases by 6.
Quality increases by 5.
Your Fulfillment effect fades.
You create 4 circles of dodo leather.

Total:
101% Progress ; 43 Durability loss
6 Successful action, 1 Failures, 1 Neutral
Averages:
13.7% Progress ; 2.3 Durability loss on Successful actions
7% Progress ; 17 Durability loss on Failures
12% Progress ; 10 Durability loss on Neutral outcomes

You use Fulfillment.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 12%.
Durability decreases by 5.
Quality increases by 4.
You use Standard Synthesis.
Progress increases by 14%.
Durability decreases by 9.
Quality increases by 2.
You use Standard Synthesis. The attempt fails!
Progress increases by 6%.
Durability decreases by 15.
The harnessed Earth element becomes unstable!
You use Standard Synthesis.
Progress increases by 13%.
Durability decreases by 14.
Quality increases by 2.
Your Fulfillment effect fades.
You use Maker's Muse.
You use Standard Synthesis.
Progress increases by 14%.
Durability decreases by 12.
Quality increases by 2.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 12%.
Quality increases by 9.
The harnessed Earth element stabilizes.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 12%.
Quality increases by 14.
You use Standard Synthesis. The attempt fails!
Progress increases by 7%.
Durability decreases by 16.
Your Maker's Muse effect fades.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 14%.
Durability decreases by 5.
Quality increases by 4.
You create 4 circles of dodo leather.

Total:
104% Progress ; 76 Durability loss
4 Successful action, 2 Failures, 2 Neutral
Averages:
12.5% Progress ; 2.5 Durability loss on Successful actions
6.5% Progress ; 15.5 Durability loss on Failures
14% Progress ; 10.5 Durability loss on Neutral outcomes


Overall Averages:
18 Successful action, 13 Failures, 13 Neutral

3/5 Dodo Leather Completed, averaging 96% Progress.

44 Actions Total
Successful: 41%
Failure: 29.5%
Neutral 29.5%

13.2% Progress ; 3 Durability loss on Successful actions
5.3% Progress ; 17.4 Durability loss on Failures
12.7% Progress ; 10.1 Durability loss on Neutral outcomes

It would take 8 Successful actions to reach 100% progress.
It would take 6 Failures to go from 105 Durability to 0.
It would take 8 Neutral outcomes to reach 100% progress.

On Successful actions, for every 4.4% Progress you receive 1 Durability loss.
On Failures, for every 3.3 Durability loss you receive 1% Progress.
On Neutral outcomes, for every 1.3% Progress you receive 1 Durability loss.
#3 Oct 26 2010 at 9:09 PM Rating: Excellent
*
216 posts
You use Maker's Muse.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 9%.
Durability decreases by 5.
Quality increases by 3.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 14%.
Durability decreases by 5.
Quality increases by 3.
You use Standard Synthesis. The attempt fails!
Progress increases by 2%.
Durability decreases by 20.
You use Standard Synthesis.
Progress increases by 14%.
Durability decreases by 11.
Quality increases by 2.
Your Maker's Muse effect fades.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 10%.
Durability decreases by 5.
Quality increases by 5.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 12%.
Quality increases by 6.
You use Fulfillment.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 10%.
Durability decreases by 5.
Quality increases by 6.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 13%.
Durability decreases by 6.
Quality increases by 5.
You use Standard Synthesis. The attempt fails!
Progress increases by 3%.
Durability decreases by 18.
You use Standard Synthesis.
Progress increases by 13%.
Durability decreases by 10.
Quality increases by 2.
Your Fulfillment effect fades.
You create 4 circles of dodo leather.

Total:
100% Progress ; 85 Durability loss
6 Successful action, 2 Failures, 2 Neutral
Averages:
11.3% Progress ; 4.3 Durability loss on Successful actions
2.5% Progress ; 14 Durability loss on Failures
13.5% Progress ; 10.5 Durability loss on Neutral outcomes

You use Standard Synthesis. The attempt succeeds!
Progress increases by 10%.
Durability decreases by 5.
Quality increases by 3.
You use Fulfillment.
You use Standard Synthesis. The attempt fails!
Progress increases by 4%.
Durability decreases by 16.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 14%.
Quality increases by 8.
You use Standard Synthesis.
Progress increases by 14%.
Durability decreases by 10.
Quality increases by 4.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 13%.
Durability decreases by 5.
Quality increases by 6.
Your Fulfillment effect fades.
You use Standard Synthesis.
Progress increases by 9%.
Durability decreases by 10.
Quality increases by 1.
You use Standard Synthesis.
Progress increases by 14%.
Durability decreases by 10.
Quality increases by 3.
You use Maker's Muse.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 13%.
Durability decreases by 5.
Quality increases by 4.
You use Standard Synthesis. The attempt fails!
Progress increases by 7%.
Durability decreases by 14.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 14%.
Durability decreases by 6.
Quality increases by 5.
You create 4 circles of dodo leather.

Total:
112% Progress ; 81 Durability loss
5 Successful action, 2 Failures, 3 Neutral
Averages:
12.8% Progress ; 4.2 Durability loss on Successful actions
5.5% Progress ; 10 Durability loss on Failures
12.3% Progress ; 10 Durability loss on Neutral outcomes

You use Standard Synthesis.
Progress increases by 10%.
Durability decreases by 9.
Quality increases by 2.
You use Fulfillment.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 14%.
Durability decreases by 5.
Quality increases by 4.
You use Standard Synthesis. The attempt fails!
Progress increases by 7%.
Durability decreases by 15.
You use Standard Synthesis.
Progress increases by 10%.
Durability decreases by 9.
Quality increases by 1.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 15%.
Durability decreases by 5.
Quality increases by 4.
Your Fulfillment effect fades.
You use Maker's Muse.
You use Standard Synthesis. The attempt fails!
Progress increases by 2%.
Durability decreases by 19.
You use Standard Synthesis. The attempt fails!
Progress increases by 5%.
Durability decreases by 15.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 11%.
Durability decreases by 5.
Quality increases by 3.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 10%.
Quality increases by 6.
Your Maker's Muse effect fades.
You use Standard Synthesis. The attempt fails!
Progress increases by 2%.
Durability decreases by 18.
You use Standard Synthesis. The attempt fails!
Progress increases by 3%.
Durability decreases by 18.
You botch the synthesis!

Total:
89% Progress ; 118 Durability loss
4 Successful action, 5 Failures, 2 Neutral
Averages:
12.5% Progress ; 3.8 Durability loss on Successful actions
3.8% Progress ; 17 Durability loss on Failures
10% Progress ; 9 Durability loss on Neutral outcomes

You use Fulfillment.
You use Standard Synthesis.
Progress increases by 11%.
Durability decreases by 11.
Quality increases by 1.
You use Standard Synthesis. The attempt fails!
Progress increases by 6%.
Durability decreases by 14.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 15%.
Durability decreases by 5.
Quality increases by 3.
You use Standard Synthesis.
Progress increases by 11%.
Durability decreases by 9.
Quality increases by 3.
Your Fulfillment effect fades.
You use Maker's Muse.
You use Standard Synthesis. The attempt fails!
Progress increases by 6%.
Durability decreases by 14.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 10%.
Durability decreases by 5.
Quality increases by 4.
You use Standard Synthesis. The attempt fails!
Progress increases by 6%.
Durability decreases by 14.
You use Standard Synthesis. The attempt fails!
Progress increases by 3%.
Durability decreases by 21.
Your Maker's Muse effect fades.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 11%.
Durability decreases by 5.
Quality increases by 4.
You use Standard Synthesis. The attempt fails!
Progress increases by 7%.
Durability decreases by 13.
You botch the synthesis!

Total:
86% Progress ; 111 Durability loss
3 Successful action, 5 Failures, 2 Neutral
Averages:
12% Progress ; 5 Durability loss on Successful actions
5.6% Progress ; 15.2 Durability loss on Failures
11% Progress ; 10 Durability loss on Neutral outcomes

You use Maker's Muse.
You use Standard Synthesis. The attempt fails!
Progress increases by 5%.
Durability decreases by 16.
You use Standard Synthesis. The attempt fails!
Progress increases by 3%.
Durability decreases by 20.
You use Standard Synthesis. The attempt fails!
Progress increases by 3%.
Durability decreases by 19.
You use Standard Synthesis. The attempt fails!
Progress increases by 5%.
Durability decreases by 15.
Your Maker's Muse effect fades.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 14%.
Durability decreases by 6.
Quality increases by 4.
You use Standard Synthesis.
Progress increases by 14%.
Durability decreases by 11.
Quality increases by 4.
You use Standard Synthesis.
Progress increases by 11%.
Durability decreases by 10.
Quality increases by 3.
You use Fulfillment.
You use Standard Synthesis. The attempt succeeds!
Progress increases by 12%.
Quality increases by 13.
You use Standard Synthesis.
Progress increases by 13%.
Durability decreases by 11.
Quality increases by 2.
You botch the synthesis!

Total:
80% Progress ; 108 Durability loss
2 Successful action, 4 Failures, 3 Neutral
Averages:
13% Progress ; 3 Durability loss on Successful actions
4% Progress ; 17.5 Durability loss on Failures
12.7% Progress ; 9.5 Durability loss on Neutral outcomes


Overall Averages:
20 Successful action, 15 Failures, 12 Neutral

2/5 Dodo Leather Completed, averaging 93.5% Progress.

47 Actions Total
Successful: 42.5%
Failure: 32%
Neutral 25.5%

12.3% Progress ; 4.1 Durability loss on Successful actions
4.3% Progress ; 14.7 Durability loss on Failures
11.9% Progress ; 9.8 Durability loss on Neutral outcomes

It would take 9 Successful actions to reach 100% progress.
It would take 7 Failures to go from 105 Durability to 0.
It would take 9 Neutral outcomes to reach 100% progress.

On Successful actions, for every 3% Progress you receive 1 Durability loss.
On Failures, for every 3.4 Durability loss you receive 1% Progress.
On Neutral outcomes, for every 1.2% Progress you receive 1 Durability loss.
#4 Oct 26 2010 at 9:32 PM Rating: Good
**
263 posts
Good work, a preamble on your interpretation of the data would be nice though.

From your tests, are you suggesting no significant differences between crafting gear vs naked?

Edited, Oct 26th 2010 11:34pm by sylph19
#5 Oct 26 2010 at 9:36 PM Rating: Decent
Avatar
***
1,330 posts
One tiny problem:

Dodo Leather ALSO has Tanning Training as a factor, just like the Common Tannery. Your testing isn't taking into account that having or not having these may be skewing the results just a tad. Wouldn't it be better to pick something that doesn't have any extra "optional" baggage? I know spetches and straps don't, but not sure what else don't.
#6 Oct 26 2010 at 9:53 PM Rating: Good
Avatar
***
3,564 posts
Ravashack wrote:
One tiny problem:

Dodo Leather ALSO has Tanning Training as a factor, just like the Common Tannery. Your testing isn't taking into account that having or not having these may be skewing the results just a tad. Wouldn't it be better to pick something that doesn't have any extra "optional" baggage? I know spetches and straps don't, but not sure what else don't.


Whether or not the item recommends additional training shouldn't matter as long as he sticks to the same exact item and doesn't change anything but his gear in between. With the training manual, even if it boosts his success rate it should boost it across the board as long as nothing else changes so the results should be roughly the same.
____________________________
FFXI (Retired)
Naiya | Mithra | Windurst | Rank 10 | 1st Lieutenant | Ragnarok
WAR: 75 MNK: 75 SAM: 66
RotZ: O | CoP: O | ToAU: O | Apoch Nigh: O

WoW
Naiyah | B.Elf | 85 Rogue | Stormscale
Doubleday | B.Elf | 85 Paladin | Stormscale


[ffxivsig]367614[/ffxivsig]
#7 Oct 26 2010 at 9:58 PM Rating: Decent
Avatar
***
1,330 posts
MasterOutlaw the Irrelevant wrote:
Ravashack wrote:
One tiny problem:

Dodo Leather ALSO has Tanning Training as a factor, just like the Common Tannery. Your testing isn't taking into account that having or not having these may be skewing the results just a tad. Wouldn't it be better to pick something that doesn't have any extra "optional" baggage? I know spetches and straps don't, but not sure what else don't.


Whether or not the item recommends additional training shouldn't matter as long as he sticks to the same exact item and doesn't change anything but his gear in between. With the training manual, even if it boosts his success rate it should boost it across the board as long as nothing else changes so the results should be roughly the same.


The concern is that not having the training manual may be impacting other factors we're not aware of, like how much of a durability hit he takes, or even having him take one when he wouldn't if he had it.

Either way, I'm not a very firm believer that +Craftsmanship increases success rates ATM. I do believe that it increases the amount of quality generated though, and would like to point out that the first set of synths has a net total of 157 Quality gained (even if some of the items busted) while the second set has a total of 127 Quality gained.
#8 Oct 26 2010 at 10:05 PM Rating: Decent
**
518 posts
Quote:
OP #'s


While the "old college" try is commendable....what you have is data that is not even worth talking about.

There are ways to test this data, what you did is not a test. It is throwing **** at a wall and calling it art.

You have no hypothesis to test, you have no theory...you have no isolation other than using DoDo skins. You used abilities that affect your "chance" of success by some "x%"

To anyone else who reads this and is thinking i should test something...Please google research methods, problem statement, scientific method, statistics, and methodology.

Don't post a bunch of numbers and say you have evidence of something cause you don't. It'd be like me saying that Zam keeps away tigers because every time my self or my friend are online reading Zam no tigers attack, and since tigers are in India and India doesn't read Zam it must be true....

Also don't use % values when dealing with numbers in the double digit comparisons, a comparison of 10/20 isn't as significant as 50% of the time...yet that's what you stat as "proof"
#9 Oct 26 2010 at 10:14 PM Rating: Default
**
263 posts
EzellLangor wrote:
Quote:
OP #'s


While the "old college" try is commendable....what you have is data that is not even worth talking about.

There are ways to test this data, what you did is not a test. It is throwing sh*t at a wall and calling it art.

You have no hypothesis to test, you have no theory...you have no isolation other than using DoDo skins. You used abilities that affect your "chance" of success by some "x%"

To anyone else who reads this and is thinking i should test something...Please google research methods, problem statement, scientific method, statistics, and methodology.

Don't post a bunch of numbers and say you have evidence of something cause you don't. It'd be like me saying that Zam keeps away tigers because every time my self or my friend are online reading Zam no tigers attack, and since tigers are in India and India doesn't read Zam it must be true....

Also don't use % values when dealing with numbers in the double digit comparisons, a comparison of 10/20 isn't as significant as 50% of the time...yet that's what you stat as "proof"


Yeah alright mang, we should all take courses in experimental design and analysis before posting any observations. Why bash someone for posting their observations? If you're knowledged in experimental design then your add in your contributions so further tests can be improved, that is how science works.

Regarding this comment
Quote:
You have no hypothesis to test, you have no theory...you have no isolation other than using DoDo skins. You used abilities that affect your "chance" of success by some "x%"

On the contrary because he wasn't testing these variables, keeping them the same causes less variance. So, it isn't even a design flaw.

Edited, Oct 27th 2010 12:16am by sylph19
#10 Oct 26 2010 at 10:28 PM Rating: Good
**
518 posts
sylph19 wrote:

Regarding this comment
Quote:
You have no hypothesis to test, you have no theory...you have no isolation other than using DoDo skins. You used abilities that affect your "chance" of success by some "x%"

On the contrary because he wasn't testing these variables, keeping them the same causes less variance. So, it isn't even a design flaw.

Edited, Oct 27th 2010 12:16am by sylph19


He added two unknown values of success to the equation. Those applied bonuses could be applied randomly or universally, we have no idea, a successful synth may have been a break had he not used fulfillment, a +15% quality could have been a +17% with muse, we don't know. Just because he used them across the board doesn't mean that the effect is universal across the board.

To truly understand the effect of +1 craftsman ship on crafting you need to be a level 1 synthier synthing a level one synth naked. Then the only variable will be +1 craftsmanship +0 craftsmanship and SE's RNG, which given enough samples we can account for.

Using Muse and fulfillment is like discounting steroids as a factor in baseball stats from 1990-2005. Well since you used steroids in all the stats and all the years throughout the sample you account for steroids...even though 80% of the stats don't have steroids affecting them....and how many "extra" home runs where hit we'll never know...
#11 Oct 26 2010 at 11:01 PM Rating: Decent
**
263 posts
EzellLangor wrote:
He added two unknown values of success to the equation. Those applied bonuses could be applied randomly or universally, we have no idea, a successful synth may have been a break had he not used fulfillment, a +15% quality could have been a +17% with muse, we don't know. Just because he used them across the board doesn't mean that the effect is universal across the board.

To truly understand the effect of +1 craftsman ship on crafting you need to be a level 1 synthier synthing a level one synth naked. Then the only variable will be +1 craftsmanship +0 craftsmanship and SE's RNG, which given enough samples we can account for.

Using Muse and fulfillment is like discounting steroids as a factor in baseball stats from 1990-2005. Well since you used steroids in all the stats and all the years throughout the sample you account for steroids...even though 80% of the stats don't have steroids affecting them....and how many "extra" home runs where hit we'll never know...


No, most experiments contain unknown variables; under the assumptions they are present in all treatments then it does not matter. Similarly, your steroid example doesn't make any sense. If you are testing a variable amongst players under the assumption they are all using steroids then steroid use does not confound the experiment. Sounds to me like you could do a bit of reading up on experimental design yourself.
#12 Oct 26 2010 at 11:33 PM Rating: Good
*
66 posts
Agreed that this isn't a great test. Problems with the design:

- Poor choice of material. Dodo Leather has a facilities requirement AND a related skill. The combination of the two makes it look harder than it probably is.

- Because of that, we don't really know WHAT difficulty this synth is. I succeeded at this same synth at rank 13, with basic guild support, over two dozen times, with only a handful of failures. That argues pretty highly that the "true" rank of the synth isn't really 19. But what is it, then?

- Realistically, we don't have a good way to calibrate the "true" rank of any synth. A good way to go about it would be to analyze a bunch of synths done at related levels, look at their success percentages, and see if any trends pop out. However, for dodo leather, how do we disambiguate "how hard is this synth" from "how much extra am I getting for the guild support" and "how much does not having the training cost me"?

- We could calculate those things based on observations once we had a baseline, but we're not anywhere near that yet - we're still wondering about how stats affect crafting!

So let's focus on designing a good experiment. What would it involve?

First we'd need to think about the different factors that Craftsmanship might affect. Here's a few, add your own if you think them up:

- Chance of succeeding on a standard attempt.
- Chance of succeeding on a rapid/bold attempt.
- Chance of the orb coming up a particular color.
- Chance of elemental instability.
- Reduction of penalty for crafting something of higher rank than you. (Essentially, a "chance of succeeding" that only shows up on hard synths.)
- Reduction of penalty for not having the required skill.
- Reduction of penalty for not having the required facilities.
- Bonus to completion percentage on a successful/meh/failed attempt.
- Bonus to quality on a successful/meh attempt.
- Reduction of durability loss on a successful/meh/failed attempt.
- Chance of a special ability becoming available on the next attempt.
- Chance of a high-quality result. (Obviously affected by quality, but not ONLY by quality?)
- Chance of an elemental instability stabilizing (or reduction of chance to go chaotic?)
- Exp gained from the craft (okay, I'm reaching here).

We have three stats, Control, Craftsmanship, and Mag. Craftsmanship; any of these could end up affecting one or more of the above.

So let's think about what we could test. Hypothesis: more craftsmanship increases the chance of succeeding on a standard attempt. To test it, we should really pick three different kinds of synths:

- One that's significantly lower than the crafter's current level.
- One that's right around the crafter's current level.
- One that's a few levels above the crafter.

We're doing this because it lets us spot a couple of trends easily. If we notice that there's a big difference in success rates for the "few levels above" craft, but not in the "right around" or "piece of cake" crafts, we can distinguish that from an across-the-board increase in chances, or no change across the board.

We pick synths that do not involve training or support. Hopefully we're picking cheap synths so that we can afford to do a bunch of 'em!

For the time being, we're not trying to determine the affect of craftsmanship on rapid or bold success rates, so we shouldn't do those. We're not trying to determine the chance that the orb will change to (x) color after a wait, so we shouldn't do that. We SHOULD keep track of attempts on white, yellow, red, and flashing orbs separately.

We're not interested in the chances of crafting under the influence of special abilities, so we shouldn't use those. The exception might be Preserve; certainly we're most interested in the effect on white-orb standard crafting in order to establish a baseline, and we don't have any reason to believe that Preserve affects the craft rate beyond turning the orb white. On the other hand, if we use Preserve, we can't use this data set to estimate chances that the orb will turn a certain color, nor chances that a special ability will become active (because we're changing both variables). Your call on whether it's a good idea, but it shouldn't affect the hypothesis in this case.

We craft a BUNCH with the recipes in question. In the event that we level along the way, we need to separate the lower-level results from the higher-level results for that particular craft. (Hopefully our "easy" and "pushing it" choices are far enough out that they're still just about as easy, or pushing it just as much, when we're done.) For each attempt, we hit "standard" without attempting to wait or use any special abilities. We write down what color the orb was, what the change in progress/quality/durability was for each attempt, and what color the orb changed to (why not?)

We punch it all into a statistical data set and run a regression. We'll need to have done three or four dozen attempts of each synth to make sure that we have enough so that our analysis of each orb color has enough data points to be statistically significant. We test the null hypothesis (i.e. "Craftsmanship has no effect on standard synth attempt success rate") and see what the numbers tell us.

Now we just need to find someone with the spare time to do it!
#13 Oct 26 2010 at 11:34 PM Rating: Decent
Scholar
**
250 posts
If people are interested in contributing to the pool of data, you could partake in this collective effort at The Syndicate
____________________________
FFXIV
Name : Kronon Reinhart
Server : Gysahl
Jobs : R24 Fisher, R23 Culinarian, R15 Gladiator

FFXI (retired)
Name : Kronon
Server : Lakshmi
Jobs : 75Drk 60Blm, 37 War/Thf/Nin/Sam
#14 Oct 26 2010 at 11:42 PM Rating: Excellent
*
66 posts
Using Maker's Mark and Fulfillment definitely change the results. Both abilities increase the percentage chance of success by an unknown amount. At the very least, they need to be looked at separately from synths that don't have those abilities active.

Furthermore, there's definitely a chance that there's a soft cap on "success rate chance" beyond which no amount of gear or ability will push chances of success (otherwise low-level stuff would really be a sure thing, and from reports, it ain't!) Increasing some synths by an unknown chance of success, but not others, makes it impossible for us to know that there's such a cap. In the worst case, we might get distorted data ("what do you mean, it actually makes it less likely?!") and come to the wrong conclusions.

It's easy to avoid this particular confounding factor; just don't do it.

There's one more point we should keep in mind - we should attempt to isolate the crafting equipment such that we're only changing one stat at a time. If you're changing your Control and Mag. Craftsmanship at the same time as you're changing your Craftsmanship, how could you conclude which of them were responsible for the change in question? This shouldn't be too hard, not many pieces of gear have multiple crafting stats on them; just remove the gear with the stat in question and leave the others alone.
#15 Oct 27 2010 at 5:14 AM Rating: Excellent
I for one hope that things like this are next on SE's plate for the Dev Q&A (among other things).
____________________________
XIV Campsite Wikibase - Community Built Camp List
[ffxivsig]400681[/ffxivsig]
#16 Oct 27 2010 at 9:25 AM Rating: Good
Avatar
***
1,330 posts
One more thing to add to your list of things to test AvatarexADV that's partially related to the success rates:

Possibility that a crafted item has a minimum stat requirement for optimal success rate, like needing at least X Craftsmanship or Y Control (or any combination of tradeskill stats) to not suffer a higher failure rate on an item.
#17 Oct 27 2010 at 2:00 PM Rating: Good
**
548 posts
My best friend was killed by a tiger while reading ZAM forums.

With regards to the OP, I don't care about your methodology, but you need a one sentence conclusion in bold print. You have obviously wasted lots of time analyzing something, and I appreciate your efforts as does the entire ZAM and XIV community...but I haven't learned anything useful from this thread, and as a result I have just wasted my own time which I do not appreciate sir.

It's as if your huge waste of time has created a black hole that somehow sucks other people's valuable time into it.
____________________________
MUTED
#18 Oct 27 2010 at 3:04 PM Rating: Excellent
*
216 posts
DonFlamenco, read the quote with the bold Results: title in the original post.
#19 Oct 27 2010 at 3:16 PM Rating: Good
***
1,636 posts
I do like that people are working on figuring this stuff out. But most testing I've read on this forum hasn't had enough trials to determine any significant results.

A couple suggestions you may want to consider if you continue your tests.

You need more tests per condition. having only 5 trials gives a huge margin of error, I would recommend at least 20 per condition.

I would test more than 1 item, but preferably at the same level. Also neither should require any special trainings. Also use the same crystals (and numbers if possible) to minimize the effects of elemental alignments.

No abilities should be used, they skew results. thats what they're there for.

If possible, use 3 different stat conditions, such as 0, 30, 50 etc.

It would also be nice to separate control, craftmenship, and magic craftmenship, but I think thats pretty hard to do, most items give multiple stats.



I know its alot, but determining real statistical significances is hard. I have nothing but respect for the players who have the patience to do it.
____________________________

[ffxivsig]971172[/ffxivsig]
#20 Oct 27 2010 at 3:34 PM Rating: Decent
Scholar
**
386 posts
I thought it was pretty clear that craftsmanship relates to quality.

You can test with Epiphany. http://ffxiv.zam.com/en/ability.html?ffxivability=29545

Unless they are being tricky with the wording and by "improved craftsmanship" they don't mean "bonus to craftsmanship skill"...
This forum is read only
This Forum is Read Only!
Recent Visitors: 21 All times are in CST
Anonymous Guests (21)