Notes on jams and jam etiquette (by Edward I. Pollak, Ph.D.)
Appropriate jam etiquette should ALWAYS be observed. If you’re a novice, stay in the background & play quietly until you get the hang of it. (This is the musical equivalent of "lurking.") No one is impressed by a newcomer (or old timer) who insists on playing over everyone else’s vocals and breaks. Rules of etiquette tend to differ from jam to jam and especially between Old Timey and BG jams. In BG jams, all pickers are expected to vamp or chop or play back up licks behind the vocalist or whichever instrument is given the nod to take a solo break. In Old Timey (OT) jams, it’s common for all banjos and all fiddles to play the melody in unison. This behavior would quickly make you persona non grata at a BG jam. Many OT jams frown on banjo players with finger picks (and possibly resonators) because such instruments overpower the more traditional-style pickers. Playing Scruggs style at some OT jams is liable to get you ridden out of town on a (f)rail. Some "Folky" jams are not jams at all but "open circles" where participants take turns singing and playing. It always pays to stay in the background for a half hour or so until you can deduce the rules. BG jams will often welcome an OT banjo player and even offer him/her solo breaks but you must obey BG etiquette and not keep frailing, etc. over other people’s breaks.
We suggest you read the rest of the etiquette article on Ed's site.