Passport for kids and infants require birth certificate and the baby's name must be mentioned on the certificate. The marital status on the passport of either the father or the mother (or both :)) must be changed to "Married" - which means, at least one of the parents must update their passport with marital status. I recommend that the mother passport be changed to update marital status as well as the name of the spouse on the passport.
To apply for passport for online, first you will need to register with the offical passport website. http://passportindia.gov.in
As a next step you need to fill the application form online and follow the instructions given on the site. Once you fill the application form, you will be alloted a time-slot (appoitment) in a regional passport seva kendra near to your location.
Kindly note, there will not be any police verification for infants'/kids' passport.
Go to passport office with the above mentioned documents of kids (other minor documentation, like photo of the baby, etc..) and submit them.
NOTE: Though photographs are taken in the passport office itself, you need to carry photographs of kids whose age is less than two years, since it is very difficult to take pictures of infacts, bringing the photographs with you is mandatory.
Fee structure for passport application has been listed here.