I found that it has to do with the presence ID that's being set to the IVR, there is no notify "terminated" being sent when calling the IVR from a internal ext.
If I'm going to the IVR -> XML, and I comment out the <action application="set" data="presence_id=xxx@xxxxx"/>, it will behave as usual.
I guess presence_id is being added there to be able to monitor the IVR, is that a bug that when dialing the IVR from a internal ext., there is no notify terminated being sent, or there is a way to make it work without commenting out the presence_id?