Line a 9x9-inch baking dish with parchment paper, allowing the paper to extend up the sides, and lightly spray with nonstick cooking spray.
In a small bowl, mix banana flavoring and yellow food color gel; set aside.
In a heavy-bottomed saucepan over medium heat, combine butter, half and half, heavy cream, and vanilla flavoring; stir continuously until the butter melts.
Add granulated sugar and stir as the mixture comes to a simmer, about 6-7 minutes; then simmer for an additional 3 minutes before removing from heat.
Add white baking chips to the saucepan, cover, and let sit for 3-4 minutes; stir until melted and smooth.
Stir in marshmallow fluff until fully incorporated.
Mix in the banana flavoring and yellow food color mixture until the color is uniform.
Pour the fudge mixture evenly into the prepared baking dish; sprinkle crushed Nilla wafers on top if desired, then cover with aluminum foil.
Let the fudge set at room temperature for 30 minutes, then refrigerate for at least 4 hours.
Once set, lift the fudge out using the parchment paper and cut into 25 pieces.