Skip to content

Querying Cloud Bigtable data #6417

Description

@timbogatchev

I am trying to follow these steps to query Bigtable data using a permanent table:
https://cloud.google.com/bigquery/external-data-bigtable#permanent-tables

The approach works quite well using the CLI, but I now have to implement it as part of our Java service.
Essentially, this is what I want to accomplish:

  public Table createExternalTable(
      String datasetName,
      String tableName,
      String sourceFormat,
      String sourceUri,
      Object columnFamilies) {
    TableId tableId = TableId.of(datasetName, tableName);
    FormatOptions bigTableFormatOptions = FormatOptions.bigtable(); 
    //Issue 1: need to give this guy columnFamilies...

    ExternalTableDefinition externalTableDefinition =
        ExternalTableDefinition.newBuilder(sourceUri, Schema.of(), bigTableFormatOptions)
        .build();
    //Issue 2: I need to provide a schema for the builder, but I don't have/need one
    //Issue 3: How do i set sourceFormat to BIGTABLE? No setter is provided by the builder

    TableInfo tableInfo =
        TableInfo.newBuilder(tableId, externalTableDefinition)
            .build();

    return bigquery.create(tableInfo);
  }

To summarize, there are 3 issues I am trying to solve:

  1. Provide columnFamilies to the FormatOptions (Or BigtableOptions). This is critical, I cannot create my table without this...
  2. Init a builder without having to provide a schema. For now I am just passing an empty one.
  3. Pass a sourceFormat to the ExternalTableDefinition, this is also critical and required.

I have spend a very long time sifting through the Google source code, but there does not appear to be a way to accomplish this currently.
Is there another way to create an ExternalTableDefinition programmatically?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

api: bigqueryIssues related to the BigQuery API.priority: p2Moderately-important priority. Fix may not be included in next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions