Suppose I have a button nested in the Template of a ListBoxItem, can I set the IsMouseOver property of the button to true, so that it looks like its moused over?
Just for illustration, the notifications on the top of the window are what I am referring to. Its basically ListBoxItem‘s with a TextBlock and Button

Unfortunately, no. “IsMouseOver” is readonly.
I’m assuming, though, that you have a custom control template for the Button, right? If that’s the case, one workaround is to mess with the Tag property of the button. Add a trigger to the ControlTemplate that gets fired when a specific Tag value is set. Then, in the DataTemplate for your ListBoxItems, just set the button’s Tag to that specific value when IsMouseOver on the item is true. Below is an example: