Question

I have a lot of xsd files with different complex types on it. I want to import data into my oracle database, but amount of data so huge and i can’t use xsd2db or altova xmlspy because it’s blowing my mind. I’m looking for simple and useful etl tool which can help me with it. Does anyone know gui tool to generate ddl by xsd?

Answer 1

This is a follow up to my comment; I am not positioning this as an answer, but it should help you understand more what is it that you are after, and maybe what you can do about it. For sure, it should be a good example for @a_horse_with_no_name…

I am not familiar with xmlspy, but given what I saw in xsd2db, it made me think of the .NET ability to infer a DataSet from an XML Schema. While the authoring style of the XSD itself may affect the way a DataSet is derived, it would be mostly insignificant for larger bodies of XSD. Even more, there is a big chance that the derivation might not even work (so many limitations there actually).

From my own experience, the derivation process in .NET gives you a very normalized structure. To illustrate, I am going to introduce a sample XML:

<ShippingManifest>
    <Date>2012-11-21</Date>
    <InvoiceNumber>123ABC</InvoiceNumber>
    <Customer>
        <FirstName>Sample</FirstName>
        <LastName>Customer</LastName>
    </Customer>
    <Address>
        <UnitNumber>2A</UnitNumber>
        <StreetNumber>123</StreetNumber>
        <StreetName>A Street</StreetName>
        <Municipality>Toronto</Municipality>
        <ProvinceCode>ON</ProvinceCode>
        <PostalCode>X9X 9X9</PostalCode>
    </Address>
    <PackingList>
        <LineItem>
            <ID>Box1</ID>
            <Load>1-233</Load>
            <Description>Package box</Description>
            <Items>22</Items>
            <Cartons>22</Cartons>
            <Weight>220</Weight>
            <Length>10</Length>
            <Width>10</Width>
            <Height>10</Height>
            <Volume>1000</Volume>           
        </LineItem>
        <LineItem>
            <ID>Box2</ID>
            <Load>456-233</Load>
            <Description>Package box</Description>
            <Items>22</Items>
            <Cartons>22</Cartons>
            <Weight>220</Weight>
            <Length>10</Length>
            <Width>10</Width>
            <Height>10</Height>
            <Volume>1000</Volume>           
        </LineItem> 
    </PackingList>
</ShippingManifest>

Conceptually, its structure is very simple: a shipping manifest entity, a customer, a shipping address and a packing list.

Converting this to an ADO.NET DataSet is a straight forward exercise, with a very clean output.

It should be easy to imagine how the number of entities (tables in your database if you wish) could mushroom for just a bit more complex XML…

As a sidebar, if one designs the XSD keeping in mind a process involving DataSets, then removing the PackingList element and moving the LineItem collection as repeating under ShippingManifest gives a somewhat simplified layout: one without the PackingList entity.

Automatic tools to convert an XSD data model to a relational model, such as .NET’s, are typically designed to generate a highly normalized structure. The denormalization, I guess, is left to the user for obvious reasons.

QTAssistant‘s XML Builder is different. Our requirement was to create an ER model which would work where .NET’s XSD to Dataset doesn’t, and with an output containing smaller number of entities, where possible. This is what QTAssistant generates for the same:

What QTAssistant did here, was to merge all entities engaged in a one-to-one relationship. From a modeling perspective, it’s an obvious sin. It does have its benefits, particularly for our users interested in a simple structure capable of capturing data (test data to be more specific).

The generated mapping (XSD to ER) is bi-directional. It means that it can be used to generate valid XML from a database, or “shred” XML data into a database (the shredding is done by generating DML statements). The way this technology is used: test cases are stored in Excel spreadsheets, XML is generated, sent to a web service, and the results are stored back in Excel.

We also generate an XML file describing the structure which through an XSLT could be converted to DDL. And this is where things could get messy, depending on you schema. It is rather common to see XSDs where simple types are unconstrained: strings without maxlength, or using patterns without a maximum length; unconstrained decimals, etc. These are just some of the reasons why, in our case, we don’t have an out of the box-straight-forward-way to generate the DDL but rather provide hooks for customizations.

So, to close my comment, I pretty much know what you want to do (I have to assume that other things, such as Oracle’s XML capabilities, or XML databases and XQuery, etc. have been ruled out). Unfortunately, the XSD really matters here, so if you can share those as per my comment, I can take a look – it’ll be up to you how much you want to share back here.