Nothing bad has happened yet after deleting the file. Memories are still there, but the images for them are gone (all the pictures). When I went back to Egypt, my house and all my stuff is still there, but the image of the house was missing when it asked me to choose my home lot. Perhaps it regenerates this. I haven't tested much.
I deleted 200 memories from one of my sims and the size of the TravelDB.package file stayed the same, so I guess it didn't actually delete anything from that file. I will play around after deleting the file and see what happens.
P.S. This game was started after I installed Pets, so it is using the latest update, unless there is a post-pets patch I don't know about.