Soak the clams in fresh salted water for 20 minutes so they expel their grit. After 20 minutes, scrub the shells with a bristled brush and place them into a clean bowl and cover with a damp towel until you need them.
Combine bacon and 1/4 cup water in a heavy-bottomed stock pot or Dutch oven over medium heat, stirring occasionally, until water has evaporated and pork has begun to brown and crisp in spots, about 8 minutes. Lower the heat to medium low and continue to cook, stirring often until the bacon is browned and crisp. Remove with a strainer spoon and drain on a paper towel lined plate, keeping the bacon fat in the pan.
Add butter, onion, and celery. Continue to cook, stirring occasionally, until onions are softened but not browned, about 4 minutes longer. Add clam juice or water and stir to combine.
Add clams and increase heat to high. Cover and cook, opening lid to stir occasionally, until clams begin to open, about 3 minutes. As clams open, remove them with tongs and transfer to a large bowl, keeping as many juices in the pot as possible and keeping the lid shut as much as possible. After 8 minutes, discard any clams that have not yet begun to open.
Add milk, potatoes, bay leaves, 1/2 the bacon and a pinch of salt and pepper to the pot. Bring to a boil, reduce to a bare simmer, and cook, stirring occasionally, until potatoes are tender and starting to break down, about 15 minutes.
Meanwhile, remove meat from inside the clams and roughly chop it. Discard empty shells. Transfer chopped clams and as much juice as possible to a fine-mesh strainer set over a large bowl. Let clams drain, then transfer chopped clams to a separate bowl. Set both bowls aside.
Once potatoes are tender, pour the entire mixture through the fine-mesh strainer into the bowl with the clam juice rapping the strainer with the back of a knife or a honing steel to get the liquids to pass through. Transfer strained solids to the bowl with the chopped clams. You should end up with a white, semi-broken broth in the bowl underneath, and the chopped clams, potatoes, salt pork, and aromatics in the separate bowl.
Use an immersion blender to blend until smooth and emulsified, about 2 minutes. If you use a regular blender be extremely careful and do not close it up fully because the steam will create pressure inside and splatter the liquid all over. Return liquid and solids back to Dutch oven. Add heavy cream and stir to combine. Reheat until simmering. Season well with salt and pepper. Serve immediately with oyster crackers and the rest of the bacon.