I was the same. Back in the day, when I rode sport bikes, I wore a full face helmet, leather and gloves, all of the time. I took a break from riding and picked it up again when I got the VTX. The I rode with a mesh riding jacket, or leather, gloves and a modular / full face helmet. As time progressed I slowly eased up on AGATT (https://www.urbandictionary.com/define.php?term=AGATT
), and went to a half helmet and often rode in jeans and T-Shirt, sneakers. Because of the wind noise with the half helmet, I now wear a quality (Shoei) 3/4 helmet and often ride bare sleeved wearing jeans and sneakers. I almost always wear gloves, not because I might wreck, although we all know that is a distinct possibility every time you ride, but because a grasshopper hurts like a bugger off the knuckles.
On our ride to Sturgis in 2014, I rode the VTX without a helmet once. It felt ... weird. I wasn't comfortable with it, even though at the time I rode with a half helmet. IMHO you should ride however you are most comfortable. Although I completely advocate always wearing a helmet, I do not endorse it's legislation. The province to our east has tried and is trying to legislate AGATT for bikers, and it is a mine field of legislation. It seems ridiculous that one can legislate what type of jackets or gloves or footwear are appropriate or such are "required" at all. Either way I don't believe in a nanny state and I don't believe in such legislation.
Just a heads up for everyone as well. This is another down side to government funded health care. Once the government controls health care, it becomes a pivotal point in any discussion involving injury mitigation. After all it's not just the individual that suffers exposure to risk anymore. It's a slippery slope my friends and liberty often comes at the expense of dollars. It shouldn't but when they can make it about cash, then liberty comes second.