I'm not certain as to how I should formulate this question; it might be considered a soft question. I am interested in finding a general way to take a category $\mathbb{C}$ and an (elementary) topos $\mathcal{E}$, which we have some chosen semantics for, and constructing a semantics for $\mathbb{C}$ in terms of that of $\mathcal{E}$. For example, suppose we are presented syntactical data for a category $\mathbb{G}$ equivalent to $\mathbf{Grp}$, and we let $\mathcal{E}=\mathbf{Set}$. Then, if we didn't have the axioms of a group object internal to the category of sets, is there some way we might realize the objects of $\mathbb{G}$ as objects of $\mathbf{Set}$ with extra structure, and morphisms as functions respecting this structure? Does it suffice to find a forgetful functor into our semantic topos, and see how it treats objects and what sort of data is forgotten?
I'm guessing that there can be no way to do this given any choice of category and topos, but are there restrictions we can place on each such that such a "relative semantics" can always be built?
Any recommendation of literature that regards this question would also be greatly appreciated. Thank you.
Note: I am not sure that a topos is necessary here. We just happen to have semantics for lots of pretty categories in terms of $\mathbf{Set}$ because it is a nice category to work with in the first place. Perhaps $\mathcal{E}$ can be any category with a semantics chosen?