Here you are making the same mistake Christian did when reading "Father" word in Bible , relation could be established with an entity out of respect
Also If you cannot explain any thing or does not have a proof it does not mean text is wrong ; it means we do not know and that is okay?
19. “Sister of Aaron” may either mean that Mary had a brother of the name of Aaron, or it may mean that she belonged to the family of Prophet Aaron. The first meaning is supported by a tradition of the Prophet (peace be upon him) and the second is plausible because that is supported by the Arabic idiom. But we are inclined to the second meaning, for the wording of the said tradition does not necessarily mean that she actually had a brother named Aaron. The tradition as related in Muslim, Nasai, Tirmizi, etc. says that when the Christians of Najran criticized the Quranic version of stating Mary as the sister of Aaron before Mughirah bin Shubah, he was not able to satisfy them, because Prophet Aaron had passed away centuries earlier. When he presented the problem before the Prophet (peace be upon him), he replied: Why didn’t you say that the Israelites named their children after their Prophets and other pious men? That is: You could have answered their objection like this as well." See also (E.N. 32 of Aal-Imran).