The egg is a symbol of birth or rebirth.  The rabbit, especially young bunnies, is also a symbol of new birth or rebirth.  It is also a symbol of fertility.  So how did they get associated with Easter?  Much of todays Easter traditions, including the name "Easter", were borrowed from pagan religions.  That doesn't mean they are evil or satanic.  The idea that all religion prior to the birth of Jesus is evil is ridiculous.  
    When Christianity first arrived in western Europe, it found that the people already celebrated a holiday at about the same time as the Resurrection celebration.  This pagan holiday was Easter, the holiday of Ester or Aester, the goddess of spring.  The celebration including having kids collect eggs and bunnies.  The eggs were painted in bright colors (why is unknown but was common in many spring celebrations).  The rabbits were sacrificed to Ester.  The adults then dressed in their best clothes and had a feast.
    Sounds familiar, doesn't it?  Christianity adopted most of it except, fortunately, the sacrifice of bunnies somehow turned into the Easter Bunny bringing the colored eggs.