Yeah, I think that as long as you realise that it's not your idea, even if you don't remember where you got it from, you can simply go "not mine either". In the past I credited several authors in my author's notes, if I remembered that what I used was their idea.

The idea that made me ask this question in the first place was so vague though and so essential to the other author's fic that I wasn't sure how to go about it and ended up asking her whether taking that part and using it was alright. I did get permission so I'm really glad I took the time to ask.