BEST - BSA Study

[ItsBrian]

https://bsabeststudy.org/

“The Building Evidence in Scouting Together Study will help the Boy Scouts of America (BSA) make Scouting the BEST program it can be.

 

When participating in BSA programs, Scouts learn and become leaders through interactions with each other and adult leaders. Scouters’ beliefs and actions are shaped by their training and experiences.

 

We will learn more about the Scouting experience through surveys and interviews conducted with thousands of Scouts and adult volunteers and through observations of BSA trainings.”

 

My troop was recently invited and plans on participating in this. Anyone else get invited?

[qwazse]

Not to my knowledge. Typically, to control costs, these intensive surveys rely on random sampling. So it's likely yours is that one in a hundred troop.

[ItsBrian]

Just now, qwazse said:

Not to my knowledge. Typically, to control costs, these intensive surveys rely on random sampling. So it's likely yours is that one in a hundred troop.

I believe the website said it’s inviting 2,500 troops.

[sst3rd]

That's nice.

sst3rd

[Saltface]

I received an email announcing the study a few weeks ago. I don’t know if that means we will be invited or not. 

Edited February 6, 2019 by Saltface

[shortridge]

I was in an SM Specific training last year with two researchers from Montclair, one of the partners, sitting in to observe and ask questions. They were doing the same with a series of trainings around the region and had even taken Wood Badge.

Interestingly, they were both registered with the BSA and wearing the field uniform, I presume to better blend in and not skew the group dynamics. Unfortunately, I was the only “normal Scouter” aside from the trainer who showed up to the training. Three people ditched the day before. I guess they learned something from that, if not exactly what they were aiming for ...

 

[Eagledad]

I was part of a polled group when National was writing NYLT. Very little of the results I saw are in the course.

I'm very skeptical because my observation is that National interprets data to fit in their predetermined plan instead of using it to question the present program. Take for example First Class in the First Year commitment. National found in  their study that scouts who earn a first class rank in one year tend to stay in the troop for at least three years. So, they encouraged units to get scouts to first class in one year, even putting  that promise in writing in all the Scout Handbooks. You can guess where that went. The program switched from one of adventure where scouts can also advance, to a program of advancement and, ... well keep advancing. One of the most common Wood Badge ticket items after that promised commitment was getting all new scouts to first class quickly.

However, we found from research in our area that scouts who just stayed in the program at least one year, regardless of rank, likely stayed in the troop for at least 3 years. In fact, we found that if a scout was still active after their first summer camp, odds were they were committed for several years. We also found that the troops with higher numbers of scouts who stay in after a year had the more active fun outdoor programs. They were the better performing overall programs in our area. Scouts in those troops tended to advance to 1st Class within 14 months. Add one other statistic that the highest drop out rate in the BSA is the first year scouts. Hmmmm. So, how should a responsible district advise troops knowing all that data?  

By the way, when we looked at the numbers of first year drop out rates at a National level 20 years AFTER National instigated First Class in the First Year program along with Troop guides and age based patrols, we found that the first year drop out rates were the same. In fact, I was told that the rate was about the same since the 60s, although I have no idea where they got those results. That attacked the wrong part of the troop program, so they ended up with the same results.

So, while I'm glad National is trying to use real time data to improve the program, they have never shown me that the data results preceeded the agenda.

Yes, I know that my earned skepticism is a downer in this discussion, but consider it a warning for keeping an open mind.

Of course National could surprise me. 

Barry

[John-in-KC]

Pardon me for being foolish, but will there be control groups of non Scouts and Scouters, or is this yet another RAH RAH SCOUTING disguised as an attempt at a rigorous study?

 

is there even a hypothesis?

[shortridge]

1 hour ago, John-in-KC said:

Pardon me for being foolish, but will there be control groups of non Scouts and Scouters, or is this yet another RAH RAH SCOUTING disguised as an attempt at a rigorous study?

This sounds like an internal review of training and other programs. I’m not sure what non-Scout people would contribute.

[ItsBrian]

5 hours ago, shortridge said:

This sounds like an internal review of training and other programs. I’m not sure what non-Scout people would contribute.

I believe it is exactly along those lines. The website I provided gives more information. 

[Saltface]

There were two documents attached to the email.

https://vhcbsa.org/wp-content/uploads/2018/12/BSA-BEST-Informational-Flyer.pdf

https://vhcbsa.org/wp-content/uploads/2018/12/Letter-from-MSurbaugh.pdf

The letter from Surbaugh states that the study "will help explore and describe the Scouting experience and how it develops character for both adult volunteers and youth in Scouting."  I'm guessing (but could be wrong) that it might be a repeat of the 2005 studies about American and Scout values (https://filestore.scouting.org/filestore/marketing/pdf/02-882.pdf & https://filestore.scouting.org/filestore/marketing/pdf/02-849.pdf). I'd be intrigued to see how the poll results have changed over fifteen years.

[Jameson76]

On 2/6/2019 at 12:06 PM, Eagledad said:

I was part of a polled group when National was writing NYLT. Very little of the results I saw are in the course.

I'm very skeptical because my observation is that National interprets data to fit in their predetermined plan instead of using it to question the present program. Take for example First Class in the First Year commitment. National found in  their study that scouts who earn a first class rank in one year tend to stay in the troop for at least three years. So, they encouraged units to get scouts to first class in one year, even putting  that promise in writing in all the Scout Handbooks. You can guess where that went. The program switched from one of adventure where scouts can also advance, to a program of advancement and, ... well keep advancing. One of the most common Wood Badge ticket items after that promised commitment was getting all new scouts to first class quickly.

However, we found from research in our area that scouts who just stayed in the program at least one year, regardless of rank, likely stayed in the troop for at least 3 years. In fact, we found that if a scout was still active after their first summer camp, odds were they were committed for several years. We also found that the troops with higher numbers of scouts who stay in after a year had the more active fun outdoor programs. They were the better performing overall programs in our area. Scouts in those troops tended to advance to 1st Class within 14 months. Add one other statistic that the highest drop out rate in the BSA is the first year scouts. Hmmmm. So, how should a responsible district advise troops knowing all that data?  

By the way, when we looked at the numbers of first year drop out rates at a National level 20 years AFTER National instigated First Class in the First Year program along with Troop guides and age based patrols, we found that the first year drop out rates were the same. In fact, I was told that the rate was about the same since the 60s, although I have no idea where they got those results. That attacked the wrong part of the troop program, so they ended up with the same results.

So, while I'm glad National is trying to use real time data to improve the program, they have never shown me that the data results preceeded the agenda.

Yes, I know that my earned skepticism is a downer in this discussion, but consider it a warning for keeping an open mind.

Of course National could surprise me. 

Barry

 

Since it is a survey by BSA National Office, can they save time by going ahead and publishing the results now?  Would save a lot of time and money to go ahead and tell the minions in the trenches what they want us to know.

Your example of the First Class First Year is spot on.  The real challenge is that it is NOT rocket science to keep Scouts active.  But to justify the headquarters jobs and spend the money, they've got to do something that appears helpful.  The secret??  Have a FUN program.  Go camping, Go outside, Go do stuff.  Now the problem is how to get that implemented at every troop, many troops take themselves waaay to seriously, 75 page troop manuals, and basically make Scouting just more school.

Scouts show up looking for fun, fire, and knives.  They get cyber chip, comparing the cost of items at three locations, and having to explain utilities to their house.  Not bad stuff, but not exactly fun, fire, and knives.

[ItsBrian]

3 minutes ago, Jameson76 said:

 

Since it is a survey by BSA National Office, can they save time by going ahead and publishing the results now?  Would save a lot of time and money to go ahead and tell the minions in the trenches what they want us to know.

Your example of the First Class First Year is spot on.  The real challenge is that it is NOT rocket science to keep Scouts active.  But to justify the headquarters jobs and spend the money, they've got to do something that appears helpful.  The secret??  Have a FUN program.  Go camping, Go outside, Go do stuff.  Now the problem is how to get that implemented at every troop, many troops take themselves waaay to seriously, 75 page troop manuals, and basically make Scouting just more school.

Scouts show up looking for fun, fire, and knives.  They get cyber chip, comparing the cost of items at three locations, and having to explain utilities to their house.  Not bad stuff, but not exactly fun, fire, and knives.

From what I’ve briefly read and heard by my SM, this is done mostly without national involved. It’s done by Montclair university and a research firm. 

Edited February 7, 2019 by ItsBrian

[Eagledad]

I assure you that National is quite involved because they are the main beneficiaries of the study results. The research professionals have the credentials and expertise to design and manage the study.  I expect Montclair will produce some publications from the study as well. 

Barry

Edited February 7, 2019 by Eagledad

[ItsBrian]

3 hours ago, Eagledad said:

I assure you that National is quite involved because they are the main beneficiaries of the study results. The research professionals have the credentials and expertise to design and manage the study.  I expect Montclair will produce some publications from the study as well. 

Barry

“How is the BSA national office involved?
The national office is providing input on the study, but the study is being conducted by external, independent researchers at MSU and AIR.”