Here's what I predict would happen if you did this: After the first day's flag ceremony, one or more adult leaders from other units will complain to the Camp Director. To avoid controversy, the Camp Director will take your SM aside and ask him to ask the boys not to wear them to flags again, since it offended some people.
I say this because I was at Philmont a few weeks ago and helped staff the second annual BSA National Roundtable (the final project for the Effective Roundtables class). One of my classmates felt it would be best if we used a little known variation on the flag cermony - moving the flags from the stage down into the audience, in which case the US flag stays on the Right hand side of the audience (rather than to the right of the podium, because by being on the floor the flag is part of the audience). This was done to make it easier for our Cub Scout age color guard to figure out where to go with the flag (personally I disagreed with this feeling, but I was alone in that). We had at lest three people who felt so strongly about this being wrong that they complained not only verbally, but also on their comment cards (quite strongly worded in fact). If, like me, you've never heard of switching the flag like this, see here: http://www.homeofher...1bfb_disp3.html
My point is... don't mess with flag ceremonies. Some percentage of Scouters will get up in arms over it.