Rose's View on Robert Browning's Poem "My Last Duchess"

I don't understand what would please this Duke. I mean, throughout this poem the late duchess is basically being slandered, when she is not even there to defend herself, by her huband, who explains that she flirted with every man she saw and didn't appreciate him. He mentions that she was happy until he put and end to her behavior and "the smiles stopped." Now I am not one to condone stepping out on a spouse, but I see nothing here that mentions infidelity; I find nothing wrong in the actions written about in this poem. I feel that the Duke is a jealous, insecure man, too engulfed with his 900 years old name to realize that women need tenderness, love, and attention. Now he's marrying another woman, after what seems like such a small amount of time after the death of his previous wife. If I didn't know any better, I'd say this late-duchess's death might not have been an accident/natural by any means- if you catch my drift.