installing VoIP in a small office?

Ok, I'm new here, but hopefully I explain my question clear enough for someone to understand. I am looking to possibly get another small office for my business. I was thinking of getting VoIP phone service for it instead of the traditional Bellsouth. I want to get 2 or 3 VoIP lines. Our high speed internet service, modem & simple 4 port router is probably going to be located in the main front office so this is presumably where my VoIP phone lines will originate from, right? My question is this....If the new business location has several other offices separate from the main office, can the VoIP lines originating from the front office be somehow also run to multi-line phone hardware in the other offices? If so, how would this be done?