Mishneh Torah, Hilkhot Tefillah 5:5
תקון המלבישים כיצד? מתקן מלבושיו תחילה ומציין עצמו ומהדר שנאמר "השתחוו לה' בהדרת קודש".
ולא יעמוד בתחילה
ולא יעמוד בתחילה
באפונדתו ולא בראש מגולה ולא ברגליים מגולות אם דרך אנשי המקום שלא יעמדו בפני הגדולים אלא בבתי הרגלים
Translation: The standard arrangement [literally: fixture] of one's clothes (when one prays), what is it?
One should arrange his clothes first, and then he should distinguish himself and beautify [literally, glorify], as it says "bow down to God in the glory of holiness". And in an ideal circumstance, he should not appear with his purse, and not with an uncovered head, and not barefoot, if the way of the people of the place is not to stand before the great without something covering their feet.
Commentary: Head covering for prayer here is connected with other ways that one "dresses up", it's a demonstration of formality and respect for God, in the same way that dressing formally (or, here, wearing shoes) displays respect for people in authority. This is wearing a hat the way that one wore a hat for all respectable occasions until sometime around the 1960s. It would be fair to say that it isn't a separate aspect of one's wardrobe that has a purely devotional purpose. We're talking about being appropriately dressed for a respectable environment.
I find it interesting, however, that the requirement for shoes and/or socks is qualified, and the requirement for a head-covering isn't. You can go barefoot (or in sandals) for prayer if people walk barefoot/sandal-shod in formal situations. But head-covering doesn't get that qualification. That takes something away from my previous paragraph- there is Something about a covered head that implies respect, even if it isn't part of the regular dress code for respectable dressing. There's something special about it that isn't getting defined here, but there's a hint. I wonder if any of the nosei keilim on this passage pick up on it... But that will have to be a post for another day.