In a saucepan, combine the sugar, flour, and salt. Add milk in gradually while stirring gently. Cook over medium heat, stirring constantly, until the mixture is bubbly. Keep stirring and cook for about 2 more minutes, and then remove from the burner.
Stir a small quantity of the hot mixture into the beaten egg yolks, and immediately add egg yolk mixture to the rest of the hot mixture. Cook for 2 more minutes; remember to keep stirring. Remove the mixture from the stove, and add butter and vanilla. Stir until the whole thing has a smooth consistency.
Slice bananas into the cooled baked pastry shell. Top with pudding mixture.
Bake at 350 degrees F (175 degrees C) for 12 to 15 minutes. Chill for an hour.