JQuery append - multiple instances after removal

JQuery append - multiple instances after removal

I am attempting to create a modal window with jquery and seem to have done
it fairly well. The only problem is when I click the contact button for
the second time it seems to append two modal boxes. One on top of the
other. And a third if clicked again....
Here is the code:
$('#contact').on( 'click', function(e){
e.preventDefault();
$('body').append('<div id="modal"></div>');
$("#modal").load('/forms/contact-form.html').hide().fadeIn(1000);
});
$('#send').click(function(e){
$('#modal').remove();
});
Can someone please explain why this is happening? Also any feedback about
what I am attempting here is welcome.