Caught Up in the World of Model Railways - Am I the Asshole?

So, here's my deal. I just started getting into model railways like a month ago. It was this thing where my neighbor, Joe, had some old ones he didn't use anymore and he showed them to me. Before I knew it, I was ordering all sorts of tracks and trains from the internet. Now, I'm pretty sure I spend way too much time on this hobby for someone who just picked it up. Anyway, my problem is that last weekend, we had a big family gathering at home. The place was buzzing with people, kids running around everywhere - you know how it goes. And there I am, sneaking into the spare room (which doubles as my model railway corner) to set everything up and start building. So yeah, I'm in this room working away on my tracks when suddenly I hear a knock. It's my aunt Linda, looking worried. She was about to tell me that everyone wanted to know where I was because it looked like I had run off or something. But she stopped short, noticed the train setup, and her face just dropped. Now, this is a woman who's always been really good at not saying what's on her mind but you could practically read her thoughts from across the room. She said something along the lines of 'Oh my, is that your new hobby?' with this tone like she was expecting me to come out and say I had bought another car or started a casino in the backyard. And then the kicker - after our brief awkward exchange where I assured her it's just a harmless little hobby, I saw her checking her watch every five seconds. She was clearly waiting for all of us to be gone so she could get in here and clean everything up. I tried to act casual and finish what I was working on but there was this constant feeling like everyone else thought I'd lost it or something. The weird thing is, at first, I was annoyed - how dare they not want me enjoying my hobby! But now that the dust has settled, I'm just wondering if maybe Linda had a point? Am I being selfish and ignoring family time for some trains? Anyway, there you have it. Model railways are eating up my life and making everyone uncomfortable. Or am I just overthinking this whole thing?