Paddling further out on a shortboard

I've been surfing for about 20 years now and in my 40s. I've always surfed shortboards. While I'm no Kelly, I'm also no Kook - hope that sums up my experience before I ask my question. I'm not aggressive in the surf and even if I get burnt i'll say nothing - although i'll give somebody a little shout if they're about to burn me, but that's more a safety thing.
I recently moved country and wondering if the etiquette here is different or I've just been a tw@t for the past few years including today.
So at my local spot on a low tide we end up with 2 peaks - outter one breaking a little bigger but then fades a bit but you can pump through into the smaller inside one. Not far apart really, especially on a small day like today.
I was out catching a few with virtually nobody else out. Eventually a couple of other guys came out. I was sitting on the outside peak, caught a wave and then proceeded to paddle back out. The 2 other guys sat more on the inside peak. I sat with them to begin with but then paddled back out to the outside peak. Each of them took one of the smaller inside waves but they sort of faded. I took the next one that came further out. To my surprise as I approached inside section one of the guys looked directly at me and just burnt me anyway. Then paddled back out to me to say he burnt me cause I was hogging the waves.
Surely, if a few guys are sitting on a crappier inside wave, as a shortboarder I shouldn't be expected to wait when there are better waves further out? I get that if I was on a longboard and always paddling out past them but surely if we're all on shortboards it's their choice to miss out on the outside waves? Or have I got this completely wrong?
I'll add that the guys were locals and even though I do surf regularly with a few locals, others still see me as an outsider. Any thoughts?
I recently moved country and wondering if the etiquette here is different or I've just been a tw@t for the past few years including today.
So at my local spot on a low tide we end up with 2 peaks - outter one breaking a little bigger but then fades a bit but you can pump through into the smaller inside one. Not far apart really, especially on a small day like today.
I was out catching a few with virtually nobody else out. Eventually a couple of other guys came out. I was sitting on the outside peak, caught a wave and then proceeded to paddle back out. The 2 other guys sat more on the inside peak. I sat with them to begin with but then paddled back out to the outside peak. Each of them took one of the smaller inside waves but they sort of faded. I took the next one that came further out. To my surprise as I approached inside section one of the guys looked directly at me and just burnt me anyway. Then paddled back out to me to say he burnt me cause I was hogging the waves.
Surely, if a few guys are sitting on a crappier inside wave, as a shortboarder I shouldn't be expected to wait when there are better waves further out? I get that if I was on a longboard and always paddling out past them but surely if we're all on shortboards it's their choice to miss out on the outside waves? Or have I got this completely wrong?
I'll add that the guys were locals and even though I do surf regularly with a few locals, others still see me as an outsider. Any thoughts?