Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

parquet-cli has an extra dependency jar #172168

Open
4 tasks done
mkmik opened this issue May 20, 2024 · 1 comment
Open
4 tasks done

parquet-cli has an extra dependency jar #172168

mkmik opened this issue May 20, 2024 · 1 comment
Labels
bug Reproducible Homebrew/homebrew-core bug

Comments

@mkmik
Copy link
Contributor

mkmik commented May 20, 2024

brew gist-logs <formula> link OR brew config AND brew doctor output

HOMEBREW_VERSION: 4.3.0-93-ge0bc557
ORIGIN: https://github.com/Homebrew/brew
HEAD: e0bc557e7b991cb23583679e1cf1c8a92b793aeb
Last commit: 3 hours ago
Core tap HEAD: 51e6e876bfa1ee5bfda14b0b9d56d9546b231c8e
Core tap last commit: 6 days ago
Core tap JSON: 20 May 11:52 UTC
Core cask tap HEAD: 8f9e1d376a3ea346f9d5b156d7a9789c73b1f2a9
Core cask tap last commit: 6 days ago
Core cask tap JSON: 20 May 11:52 UTC
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_DISPLAY: /private/tmp/com.apple.launchd.TZMewcwAfn/org.xquartz:0
HOMEBREW_EDITOR: code -g -w
HOMEBREW_MAKE_JOBS: 16
HOMEBREW_SORBET_RUNTIME: set
Homebrew Ruby: 3.3.1 => /opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.3.1/bin/ruby
CPU: 16-core 64-bit arm_palma
Clang: 15.0.0 build 1500
Git: 2.45.1 => /opt/homebrew/bin/git
Curl: 8.4.0 => /usr/bin/curl
macOS: 14.3-arm64
CLT: 15.3.0.0.1.1708646388
Xcode: 15.3
Rosetta 2: false

Verification

  • My brew doctor output says Your system is ready to brew. and am still able to reproduce my issue.
  • I ran brew update and am still able to reproduce my issue.
  • I have resolved all warnings from brew doctor and that did not fix my problem.
  • I searched for recent similar issues at https://github.com/Homebrew/homebrew-core/issues?q=is%3Aissue and found no duplicates.

What were you trying to do (and why)?

I'm trying to use the parquet CLI tool, example invocation:

parquet schema foo.parquet

What happened (include all command output)?

Exception in thread "main" java.lang.NoSuchMethodError: 'shaded.parquet.org.apache.avro.Schema org.apache.parquet.avro.AvroSchemaConverter.convert(org.apache.parquet.schema.MessageType)'
        at org.apache.parquet.cli.util.Schemas.fromParquet(Schemas.java:85)
        at org.apache.parquet.cli.BaseCommand.getAvroSchema(BaseCommand.java:396)
        at org.apache.parquet.cli.commands.SchemaCommand.getSchema(SchemaCommand.java:104)
        at org.apache.parquet.cli.commands.SchemaCommand.run(SchemaCommand.java:82)
        at org.apache.parquet.cli.Main.run(Main.java:163)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:82)
        at org.apache.parquet.cli.Main.main(Main.java:191)

What did you expect to happen?

{
  "type" : "record",
  "name" : "schema",
  "fields" : [ {
    "name" : "foo",
    "type" : "string"
  }]
}

Step-by-step reproduction instructions (by running brew commands)

* `brew install parquet-cli`
* `parquet schema foo.parquet`

Workaround

The issue can be fixed by removing an extra jar the package accidentally installs.

rm $(brew --cellar)/parquet-cli/1.14.0/libexec/parquet-avro-1.14.0.jar

The upstream ticket https://issues.apache.org/jira/projects/PARQUET/issues/PARQUET-2142?filter=allissues basically just says that the avro dependency is bundled in the main parquet-cli jar, but the signature of the method has been altered, making it incompatible with the parquet-avro jar, which gets pulled in as a dependency and included by the parquet-cli formula, but it shouldn't.

@mkmik mkmik added the bug Reproducible Homebrew/homebrew-core bug label May 20, 2024
@SMillerDev
Copy link
Member

Can't we use this as a patch? apache/parquet-java@62b774c

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Reproducible Homebrew/homebrew-core bug
Projects
None yet
Development

No branches or pull requests

3 participants
@mkmik @SMillerDev and others