Yes they will become normal Sims just like adult Sims do. I imagine it works the same way for the potions, I can check to make sure.
Edit: I did make a nooboo with the potion and they have the tombstone in their inventory so I assume that they are like the ghosts you get with death. I will feed the nooboo Ambrosia and see if it works.
Edit: It worked, the ghost baby born with a Ghost potion was resurrected with Ambrosia.