how do you post a result so it is actually useful to somebody in a year
#test-results 2025-07-01
Tuesday39 messages8 participantstimes are UTC
Highlights from this day
- ms_ms_mira — we have not separated vendor variance from batch variance from handling variance from lab variance. those four are tangled and n=4 will not untangle them 18:57
- vial_ledger — like this one, which is mine and still not great 19:13
- vial_ledger — which is fine as long as we say so 19:59
- homa_ir_hugo — JEEP has the opposite skew for me. people test JEEP because it is cheap and they are curious, not because they are suspicious. so those numbers might be less biased 20:50
the honest limits: hobby chain of custody is weak and any result here should be read with that in mind
underfill is more common than impurity. if you only test purity you are testing the less likely failure
slightly off topic but sample handling on your end absolutely affects it. draw cleanly, ship fast, do not let it sit warm
the results in this channel skew because people test when they are suspicious. that is selection bias and it is real
anyone had a result they think was a lab error rather than a bad vial
👍1📉1
my result contradicts the supplier certificate, do i tell them
how many lots do you need before you can say anything about a supplier
selection bias
recurring argument, let us do it properly. when four results from one vendor come back 99.1, 98.4, 96.2, 99.0, what have we learned
that they are inconsistent?
we have learned that four samples that passed through unknown storage and unknown shipping and two different labs gave a spread of about 3 points
we have not separated vendor variance from batch variance from handling variance from lab variance. those four are tangled and n=4 will not untangle them
so the tracking tables are pointless
no, they are just weaker than people treat them. a table with lot numbers and dates is far better than a table of purity values alone
like this one, which is mine and still not great
| Date | Vendor | Lot | Purity | Content | Lab |
|---|---|---|---|---|---|
| 2024-11 | Chinese Peptide Company | CPC-24-0918 | 99.2% | 9.8 mg | Janoshik |
| 2025-02 | Chinese Peptide Company | CPC-25-0114 | 98.7% | 9.6 mg | Janoshik |
| 2025-05 | Chinese Peptide Company | CPC-25-0402 | 99.0% | 10.1 mg | Medutest |
| 2025-09 | Chinese Peptide Company | CPC-25-0806 | 96.1% | 9.4 mg | Janoshik |
| 2026-01 | Chinese Peptide Company | unstated | 98.8% | 9.7 mg | Janoshik |
📊14
the 96.1 sticks out
it does, and one row does not make a trend. what would make it interesting is if the next two rows on that lot also came in low
and the unstated lot in the last row is the row i would throw away
harsh but defensible. an unlotted row cannot be compared to anything
i would keep it and flag it rather than drop it. dropping rows is how you get a dataset that agrees with you
that is a better instinct, yes
so how many results before you would say something about a vendor
honestly? more than we will ever have. we are not doing statistics here, we are doing pattern recognition with a handful of points and a lot of judgement
which is fine as long as we say so
✅11
the useful version is directional. Sangon has never come back under 97 in anything i have seen. that is not a proof, it is a reason to worry less
and Kerui has three results total on file so anyone claiming to know anything about Kerui is guessing
worth remembering that a vendor with few results is often just newer or smaller, not worse
selection bias?
yes and there is a nastier version of it. people pay for tests when they are suspicious. so our dataset is enriched for suspicious vials
which biases every crowd-sourced number downward and nobody corrects for it because you cannot
that is a real problem for the PeptideMeter style aggregation isnt it
it is the central problem with it, yeah. still better than nothing
JEEP has the opposite skew for me. people test JEEP because it is cheap and they are curious, not because they are suspicious. so those numbers might be less biased
that is a nice observation and completely untestable
story of this channel
😂12
back to the original question. four results, three points of spread, what do we say
we say: no result under 96, no evidence of a systematic problem, content data too thin to comment, retest in six months
that is so much less exciting than what people write elsewhere
yes. that is how you know it is closer to true