The Reluctant Neighbor

So I moved into my new apartment in this bustling city about three months ago. Everything was going well until the guy next door introduced himself a couple weeks after moving in. Nice enough, but he's also super friendly and talkative, something that doesn't exactly make me feel comfortable, especially considering I'm more of an introvert. Anyway, one day he comes knocking on my door with this big grin on his face. He brought over some sort of home-cooked meal as a 'welcome-to-the-neighborhood' gesture and insisted we sit down together to eat it. At first, I tried to refuse politely but after like five minutes of him pleading, I just gave in, what else could I do? We sat there awkwardly for about an hour before he finally left, still all smiles. It wasn't that bad, really. The food was okay and we didn't argue or anything, but... let's just say it felt very forced. Now every week, without fail, this guy shows up with something new to cook and share. I'm starting to feel like a jerk for turning him down earlier because now I can’t bring myself to tell him no again. But at the same time, spending time with someone when you don't really want to feels so disingenuous. It's eating away at me every day as I struggle with how to handle this situation. I guess I'm just looking for a way out without hurting anyone's feelings too much. Maybe there’s some easy way to let him know I’m not quite ready to be that close of friends yet? But then again, what if he thinks it's because of something he did or said and now feels rejected? Honestly, I'm just stuck in this weird place where all my efforts are making me more anxious than anything. Maybe one day I'll figure out a way out of this social trap.