Home/ Questions/Q 8406245
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-09T23:03:24+00:00

I’m at the very beginning of building a bigquery uploader in Java. Goal is

  • 0

I’m at the very beginning of building a bigquery uploader in Java. Goal is to download a full twitter stream and uploading that into hourly buckets, then processing these in Dremel for a topic detection and tracking project. This is all Java on MacOSX in Eclipse Juno, plus Maven with a local Nexus proxy.

I’m stuck at the starting gate; finding and compiling a simple Java sample that authenticates and uploads a CSV file. Closest I’ve found is bigquery-appengine-sample, although I don’t see why I need appengine for a bigquery project. Will explore that later.

Problem is, the project won’t build in maven. No error flags shown in eclipse, but often eclipse error flags are unreliable. The pom shows a red eclipse flag on this element:

com.google.apis
google-api-services-bigquery
${bigquery.version}

Maven install fails with

[ERROR] Failed to execute goal on project bigquery-appengine-sample: Could not resolve dependencies for project com.google.api.client:bigquery-appengi
ne-sample:war:1.0.0-SNAPSHOT: The following artifacts could not be resolved: com.google.apis-samples:shared-sample-appengine:jar:1.3.2, com.google.api
s:google-api-services-bigquery:jar:v2-rev18-1.7.2-beta: Failure to find com.google.apis-samples:shared-sample-appengine:jar:1.3.2 in …SOF won’t allow URLs… was cached in the local repository, resolution will not be reattempted until the update interval of nexus has elapsed
or updates are forced -> [Help 1]

Nexus proxies are defined for http://mavenrepo.google-api-java-client.googlecode.com/hg/ and http://google-gson.googlecode.com/svn/mavenrepo/ (both set as SNAPSHOT). Neither of these seem seem to be working. Browse storage and indexes both come up empty (just an archetype catalog). However Browse Remote does seem to work AFAICT; shows a maven repository tree that seems complete.

So my question: how to build bigquery samples in Java with maven?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Based on what I can observe from outside google, the answer to the question as posed is:

    • No java example is available (yet?) that covers uploading. The sample
      at the address in the posted answer only covers querying sample data
      that is already present in the account.

    • The closest sample to what I need seems to be the
      command line and python samples. But using them is complicated by the fact that
      each sample is hard-wired
      to work only with specific maven dependency versions, samples aren’t being updated
      when new API versions are released, and older API versions are often not available via
      maven.

    • The maven problem seems to originate from using a wildly idiosyncratic version
      numbering system (such as the v2beta1-rev17-1.7.1-beta-beta example
      above). Maven relies on strict adherence to convention, and such
      version numbers definitely don’t comply. For rapidly changing
      beta code, version numbers should be major#.minor#.micro-SNAPSHOT as documented in the maven book. NO EXCEPTIONS or maven breaks as explained in the comments.

    URGENT REQUEST: Please provide a working java sample that covers all BigQuery features, particularly step 1; uploading data. Please bundle the samples with the API code to ensure they stay in sync. And adopt maven conventions throughout, including retroactively.

    • 0