Linphone was surprisingly easy to modify/rebuild for iOS (didn't try android) but building the client alone won't get you working with push notifications out of the box and they've tightly coupled other features with their serve(r/ice) that make me not want to give them time/attention despite their offer of integration services (at a cost they said little ol me likely can't afford).
Not your question but on the Windows desktop I must say MicroSip is clean and the devs offer branding at a very reasonable rate.
Anything iOS or Android requires either using someone elses service (typically configured as a proxy) or coming up with your own solution for push notifications which are required for reliably receiving calls when the app isn't open.