As far as why its becoming a standard, the SVG format is viewable by web browsers.
That's because it's spec was to be used on the browsers first and foremost. Hence why it can also be viewed in plain text editors (although just in markup unless it has some extension to view otherwise).
It has some other beneficial options for a vector format that other vector formats do not such as being able to be animated.
You can animate other vector formats as well. At least Ai formats, just not by using the original program of creation (at least not as easily/robustly). The biggest difference is because it was created for web first, JS can also be used to make elements clickable or program other aspects into it as well. That's not something that I'm aware able to do with your proprietary formats.
And depending on which vector program using, it can be used to to handle/store plugin data that doesn't really do anything to the original SVG data, but can be used for exporting unrelated data files. But some programs (like Ai/Draw) delete that data when saving in those respective programs, while something like Inkscape will ignore it and it will be retained.
I think of it almost like a generic type of file like a PDF which many different kinds of software can create..
Not really. SVGs are actually a very specific file spec, a vector file, just happens to use XML markup. More like Ai/Draw files. PDFs are more container files that can house any number of data types in it. Now, it's an open spec, much like PDFs, in that sense they are the same, but it is not a generic container type of file like PDFs are.