What good are clinical trials?

[pixmedic]

I am 95% certain that 99% +/- of the adult population (in the U.S. anyway) has a choice as to whether or not they want themselves or their children to be vaccinated.
I think I can say with some degree of certainty, +/- 1%, that whether or not they base that decision on any significant research, or consults with multiple medical professionals, or even questioning the effectiveness-vs-risks of said vaccines is ultimately up to them.

 

[Gavjenks]

Now you are just being silly. Doctors have more education and experience to base their decisions on than you do with some stats. It's not just a numbers game, its a big picture thing.

1)
What exactly do you think doctors learn in medical school that makes them better able to decide from the same set of numbers which choice better fits my personal opinions on life outcomes?

Seriously, what part of the curriculum? My brother is a medical doctor and I'm pretty familiar with the different phases of his education, and I'm not aware of any part of medical school or residency that would make him more qualified than me on a decision like this.

In fact, my own psychology and philosophy degrees and subsequent years earning a living doing research have given me MUCH MORE statistics training and real world application of statistics and scientific analysis and interpretation of published studies than he got in school or in his career.

2)
Even if I wasn't a professional researcher, I would still be perfectly capable of interpreting numbers collected and presented to me in a clear way. Once it's been collected from studies and put in a tabular format, you only would need a high school education, if that, to make an informed decision from it. If you still trust doctors more and want doctors to offer their own opinion ON TOP OF giving us the raw numbers, then fine, that's okay. But I don't see any argument here for keeping the data from us. Give both.

Policies that just assume every lay person is an ignorant slob who must be shielded from any critical thinking of their own decisions are inappropriate and dangerous.

3)
You're right it's not "just a numbers game." There are a bunch of non-numbers factors involved, almost all of which I know better than my doctor on.

Because there is no "best medical decision" in almost any case in any objective sense. There are always tradeoffs in intangibles. For example, chemo buying you a few extra years, but you're going to be super sick during them, is it worth it or not? There's no objective answer to that. It depends on the patient's current amount of unfinished business in life, their philosophy, how much they care about quality vs. quantity. There's NO way for the doctor to know any of those things, which constitute the majority of the decision to be made. I know myself and my preferences infinitely better.

This is equally true of vaccines and any other procedure too. In fact, it's even MORE true of something like vaccines, because death and complications are so much less likely than with chemotherapy, that we have much less data on the complications. So it's an even greater %age of gut instinct and philosophy and subjective junk. But for some reason they just aren't treated the same way. That's what is bothering me.

I should have been more specific. Clinical studies for OTC products is what I was referring to. Clinical trials for actual prescription drugs ad treatments are a different kettle of fish.

This is an interesting distinction which is also troubling.

How do you know whether to run something as an OTC trial versus a prescription trial, if you haven't run the trial yet, and have no idea how dangerous it might be? There is yet another layer of guesswork going on at this decision point.

It's up to the scientists to decide whether that difference is clinically significant or not.

Yes, but they don't have the training to do so. I know they don't, because nobody does, because it's mathematically / clinically / logically / in every other way impossible to extrapolate from data on 500 participants to make decisions about events which might be critically important even at rarities as low as 1/1,000 or 1/10,000. It simply can't be done. You need vastly more raw data.

It's exactly like saying "Hm, I wonder how often this die will land on a 6? Let's roll it 3 times, tabulate the results, and then make a conclusion."

 

[Gavjenks]

I am 95% certain that 99% +/- of the adult population (in the U.S. anyway) has a choice as to whether or not they want themselves or their children to be vaccinated.

For now, yes. Looking at the way things have been headed, I'm not so sure that will necessarily be the case 10 years from now.
I just now spent all of 15 seconds searching on google and already found this op ed piece, for example, from just two days ago, in the NYT:
Eliminate Vaccine Exemptions - NYTimes.com

Crazy stuff comes up like that more and more often, all the time recently. And yes it ebbs and flows with flu season, but there's also a clear trend over years as well.

This stuff matters! Think about the ridiculousness of potentially legally forcing people to inject chemicals into their arm when nobody on the planet actually knows how likely those chemicals may or may not be to kill you or disable you, or how likely they are to save you from dying or disability! Madness. Also, even stuff as simple as "what if I have a proven record of severe allergic reactions to past vaccines?" "Lolnope! Gotta take it anyway, and then proceed to go into anaphylactic shock. It's for your own good." .... Hopefully it's just some kooky op ed people, and stays there, but who knows.

Or maybe something comes out of that. And maybe it even expands outside of vaccines to random other medical treatments that strangers and/or politicians are going to decide I should have.

 

[Gavjenks]

(Sorry triple post, didn't realize heh)

Also, in my experience, for some reason medical research tends to be way more hit and miss than psychology research. Average quality is maybe similar, but medical studies are much more either awesomely done, or complete piles of shockingly abysmal BS that seem to have been designed by middle school students. For example, here's a vaccine article I looked at yesterday while thinking about this. It's not about anything too controversial, so it's good for this thread. Their claim is that the flu virus makes you more sociable, as maybe an evolutionary chemical adaptation or something. Cool concept, maybe it's true, okay, great. How did they study it?

Change in human social behavior in response to... [Ann Epidemiol. 2010] - PubMed - NCBI

Turns out that:
1) "Exposure" to the flu virus was in fact just a vaccine shot (i.e. "we are lazy and cheap and partnered with our colleague at Walgreens or whatever for a quick publication rather than thinking it through at all."). Thus,
1a) It's dead virus, not live, which totally changes your immune response details and chemical reactions, and thus doesn't tell you anything about potentially evolved natural infection strategies. Instead it can just be a random thing that happens when dead viruses are encountered from the start, but not live ones...
1b) It's injected in arm muscle, not mucus membranes. Also totally changes the immune response details and defeats the purpose of the study.
1c) It's a syringe with not just viruses but a bunch of other crap like aluminum and preservatives and whatnot. Maybe vaccine preservatives make you more sociable, for example? You'll never know since you failed to make any effort to isolate variables properly.

2) They didn't run any control group! The "controls" were the same people before and after their vaccine shot, not actual blind sham injection separate controls. Lol?
2a) Maybe you get more sociable after getting any injection!
2b) Maybe you get more sociable from exposure to fresh air during your trip to the lab and social interaction with experimenters... hurr durr.
2c) Maybe you get more sociable when you just participated in a cool study and want to go tell all your friends, so you talk to more people after because you want to tell the story...

^That's the quality of probably 2/3 of the medical studies I look up. That's the sand foundation you're basing your trust in if you rely on doctors to interpret information and make decisions for you sight unseen, even when they DO have the statistical power they would need. Again, some studies are awesomely done, but the number that are like this one clearly implies that at least a pretty decent chunk of reviewers out there thought this was great research, and that a big chunk of consumers of the research are happy with it. Add on too few participants on top of methodologies like those, and it gets way more depressing.

 

[limr]

2) They didn't run any control group! The "controls" were the same people before and after their vaccine shot, not actual blind sham injection separate controls. Lol?
2a) Maybe you get more sociable after getting any injection!
2b) Maybe you get more sociable from exposure to fresh air during your trip to the lab and social interaction with experimenters... hurr durr.
2c) Maybe you get more sociable when you just participated in a cool study and want to go tell all your friends, so you talk to more people after because you want to tell the story...

^That's the quality of probably 2/3 of the medical studies I look up. That's the sand foundation you're basing your trust in if you rely on doctors to interpret information and make decisions for you sight unseen, even when they DO have the statistical power they would need. Again, some studies are awesomely done, but the number that are like this one clearly implies that at least a pretty decent chunk of reviewers out there thought this was great research, and that a big chunk of consumers of the research are happy with it. Add on too few participants on top of methodologies like those, and it gets way more depressing.

First, a big ole facepalm at the design of that 'study.'

Second, I'm all punchy from being tired and headachy, but the 'hurr durr' somehow hit me in a way that has left me subject to uncontrollable giggles.

 

[runnah]

Well I am glad to see that Gav knows more than a trained medical doctor.

Arrogant much?

 

[pixmedic]

i know that there are different strains of the flu.
and I know that the flu vaccination is based on whatever the most prevalent strain is that season, which does not preclude you from getting another strain.
and and I know that I got the flu shot from work last year, and a few weeks later came down with the worst case of the flu I have had in years.
and and AND my partner, who did not get the flu shot, managed not to catch the flu at all.

This could entirely be coincidental. I do not know with any degree of certainty that getting the flu shot -vs- not, and the fact that I got the flu and my partner did not, are related in any way shape of form.

im just sayin. next flu season i may rethink getting the flu shot.

 

[Gavjenks]

Well I am glad to see that Gav knows more than a trained medical doctor.

Arrogant much?

I am essentially a trained study designer and interpreter full time (during the majority of semesters when I have had research grants and minimal teaching loads). So yes. I am better at interpreting studies than doctors who spend a comparatively tiny part of their time doing that, and who probably have 10% as much education in that specialty. I'm still also interested in the doctor's information they give me with regard to results of my blood work or whatever clinical stuff is relevant at the time, of course. But to the extent that their decision is based on LITERATURE, not clinical issues, I would prefer to take over for my own decisions on that front.

Plumbers are better at plumbing than landlords are who only fix pipes occasionally.
Particle physicists are better at particle physics than are high school science teachers who cover that curriculum as 5% of their semester.
Etc.


Also, when it comes to medical decisions about YOUR body that involve YOUR philosophical preferences, everybody always has an inherent advantage in making the right decisions for them over a doctor who is mostly a stranger usually. You know yourself and your philosophical opinions about medical care in general better than your doctor does, even if you're a dog walker. It is entirely unclear whether these advantages in self-knowledge do or do not on the whole override the advantages the doctors have over average patients in literature knowledge, when it comes to making a final decision on treatment. BOTH should be presented to patients so they can choose the mixture of information they will decide on:
A) The doctor's opinion
B) The raw facts that they might apply their own philosophy to in a way the doctor couldn't anticipate.

That way, if you feel like going by the doctor's opinion 100%, then you can. If the dog walker wants to go by 70% doctor, 30% their instincts about the numbers themselves, great, they can. If I want to go 50% doctor, 50% study results, then I can too. Providing the most information allows everybody the most freedom and control over their own medical care.



Note that I'm not saying people should get to decide to get MRIs just cause they feel like it, or prescribe themselves medicines. I'm only talking about information for potential NEGATIVE decisions to NOT get certain things done to your body.

 

[baturn]

OP - you have a better chance of dieing in a fatal accident on your way home from the Dr. or pharmacy than from any of the purchases you made there or shots you may have received. Your decisions are your own to make and if doing all that math makes it easier for you, go for it. All I get from it is an enormous headache, and I'm now off to the medicine cabinet without doing any research at all.

 

[Gavjenks]

Your decisions are your own to make and if doing all that math makes it easier for you, go for it.

Vaccines are a hobby and I've researched them a lot, but for most products I don't have the time to do the math.

I guess essentially what I'm suggesting is that a centralized institution of some sort collates basic data from studies, does enough of the math to provide standardized summary thresholds, and prints it on medications or on handouts for common enough procedures. Just like the nutrition labels required on the back of food that give you all the standardized collated math summaries for nutrient ratios, basically, but for medicine risks from known studies.

So instead of 300 million people wastefully attempting to or not bothering to do research independently, 5 guys do it once, and print it on the product, and there you go. the they check up on it once a year or whatever to update. Way more efficient, honest, useful. We already trust lay people to interpret and make decisions if they please on nutrition data, why not on clinical trial outcome data? You could even have those same standardized layout sorts of things: instead of iron and folate and riboflavin, it's cardiovascular, lungs/airways, brain, immune system, gastrointestinal, death, psychological, etc. best known odds.

 

[runnah]

So instead of 300 million people wastefully attempting to or not bothering to do research independently, 5 guys do it once, and print it on the product, and there you go. the they check up on it once a year or whatever to update. Way more efficient, honest, useful. We already trust lay people to interpret and make decisions if they please on nutrition data, why not on clinical trial outcome data? You could even have those same standardized layout sorts of things: instead of iron and folate and riboflavin, it's cardiovascular, lungs/airways, brain, immune system, gastrointestinal, death, psychological, etc. best known odds.

It cannot be done. There can be no standardized side effects label because each person is different. There are too many variable that come into play in how a person reacts to vaccines, drugs or food. Sure you can do a broad list of known side effects like you hear on TV but to be specific you'd have to consult a doctor.

 

[Gavjenks]

There can be no standardized side effects label because each person is different.

It's based on clinical trials, the entire concept of which are testing for population averages, not accounting individually for each person's physiology. So yes, it can be done, and is in fact already done every single day in clinical trials. All that's needed is collating that data and printing it. And yes, of course you still need a doctor to check for drug interactions and tell you about it in the first place and make sure you aren't contraindicated, etc. etc. still need a prescription. I would just want to present this data alongside their counsel. Or for OTC drugs, just in addition to the normal labels there already. Maybe also put asterisks next to condition categories especially likely to have special considerations for large groups of people (like diabetics), etc.

But roughly, something like this that I just whipped up:



If you have absolutely no data, then you just put "1" in that spot. Because you're always 95% confident that there's a lower than 1/1 chance of any side effect. Higher numbers are more reassuring. Round everything to 1 or 2 significant digits.

For prescriptions, in large part this would be a conversation facilitator for most people more so than a circumvention of doctor's suggestions.

 

[BrickHouse]

Your decisions are your own to make and if doing all that math makes it easier for you, go for it.

Vaccines are a hobby and I've researched them a lot, but for most products I don't have the time to do the math.

I guess essentially what I'm suggesting is that a centralized institution of some sort collates basic data from studies, does enough of the math to provide standardized summary thresholds, and prints it on medications or on handouts for common enough procedures.

I often use something that is called a Cochrane review. It collates large amounts of GOOD quality studies and makes recommendations based on the available evidence.

 

[runnah]

There can be no standardized side effects label because each person is different.

It's based on clinical trials, the entire concept of which are testing for population averages, not accounting individually for each person's physiology. So yes, it can be done, and is in fact already done every single day in clinical trials. All that's needed is collating that data and printing it. And yes, of course you still need a doctor to check for drug interactions and tell you about it in the first place and make sure you aren't contraindicated, etc. etc. still need a prescription. I would just want to present this data alongside their counsel. Or for OTC drugs, just in addition to the normal labels there already. Maybe also put asterisks next to condition categories especially likely to have special considerations for large groups of people (like diabetics), etc.

But roughly, something like this that I just whipped up:

View attachment 69433

If you have absolutely no data, then you just put "1" in that spot. Because you're always 95% confident that there's a lower than 1/1 chance of any side effect. Higher numbers are more reassuring. Round everything to 1 or 2 significant digits.

But you are putting data on there that is nothing more than a guess. You even showed this by saying "Actual rates etc...". All this would accomplish is to scare people.

Yes if you had 100 people, identical in every way, take product A and one died, then that would be a result. But since we all vary on a genetic and environmental level, the "1 in 100" is meaningless information. Even if the study were specific enough to say that diabetic men above 50 from Nebraska need to be wary of heart issues, you'd still don't end up with a solid result..

But at the end of the day the best way is to say, "Side effects may include blah blah blah, please see your medical doctor before taking." Boom that it, that is all the info you need to give people. Then it is up to the individual to ask their doctor, who knows all their medical issues, if the product is right for them.

 

[Gavjenks]

But you are putting data on there that is nothing more than a guess.

No, it's based on clinical trials.
Not random guesses...
These numbers are a direct relative to p values they already publish (it's not a p value, it is related to it fairly directly, though)

You could even go do this retroactively fairly easily. The larger the studies they invested in, in combination with drugs that show low side effects (compared to controls!) = smaller fractions can be guaranteed at 95% confidence.

That is purely objective, very well defined, and actually quite useful.




You SHOULD BE scared if the numbers are low, because that means they have either barely tested the thing, or it has tons of complications reported above baseline (or worst of all, both). So if people are scared by low numbers, then that's good... that's the whole point. I want to scare people away from things that are dangerous or untested... and push them toward things that are widely tested and safe.

Don't you?




Edit: if you want to give people some perspective, it's easy: just draw up the same chart for something like "driving 100 miles in your car" and a few other intuitive things, and put them up on a website linked from the label. People can go and compare to get a better sense of what the numbers mean in comparison to normal risks they undertake every day.