I like how SQLite library can be included into an application and the application has a full-fledged database to use.
Similarly, is there an open source versioning library that I can include into my application so that I can save versions of files as well as do diffs and merges?
SVN doesn’t require any prerequisites on end user machine. You can embed SVN right into you app. To learn more on subversion integration, visit ‘Application Integration/Embedding‘ thread on SVN forum.