Hey HN friends!
I've taken `fx` back to the drawing board and completely rewritten it from the ground up. Excited to share what's new:
1. *Going Big*: `fx` now gracefully handles even the most massive JSON files.
2. *A New TUI Look*: Dive deep into your data with a revamped terminal interface—now with themes!
3. *Swift Navigation with Dig Fuzzy Search*: Feeling lost in JSON? Just type `.` and navigate with ease.
4. *Powerful Regex Search*: Scan across your entire JSON content with precision.
5. *Elegant Long String Wraps*: No more cut-offs. Your strings wrap beautifully now.
6. *All Things JSON*: Added love for comments, trailing commas, and JSON streams.
Pouring my heart and soul into this rewrite has been a journey to make `fx` faster and more powerful. If you find value in what I've crafted and want to support its future, consider sponsoring on GitHub.
Would love to hear your thoughts and feedback!
> In fx, arguments are treated as JavaScript functions. Input data is passed sequentially through each provided function.
> echo '{"name": "world"}' | fx 'x => x.name' 'x => `Hello, ${x}!`'
Wow, that is so nice. Having to memorize JQ syntax is such a pain.
Would've been awesome if results are displayed under the commands. For example, command
echo '{"name": "hello"}\n{"name": "world"}' | fx '.name'
Having used https://jqlang.github.io/jq/, wondering your pitch on advantages, alternative use cases for using fx?
Not OP but
echo '{"name": "world"}' | fx 'x => x.name' 'x => `Hello, ${x}!`'
I love jq, I think it's extremely powerful, but I have to load up the documentation in a side window every time I have use it.
Equivalent jq seems okay here:
echo '{"name": "world"}' | jq '"Hello \(.name)"'Yes and both can be expressed in js and jq. Poinjt is what JS is known by much more peopel. As fx uses Node or Deno we for example can use lodash in fx:
fx '_.groupBy("commit.author.name")' '_.mapValues(size)'
'_.toPairs' '_.sortBy(1)' '_.reverse' '_.take(10)' '_.fromPairs'
Looks like the GitHub API: https://docs.github.com/en/free-pro-team@latest/rest/commits...
I'm looking forward to your jq version. I love jq and use it all the time, but for anything of that complexity, I'd probably just use Python.
As someone who doesn't use jq, a number of questions immediately jump out at me. Why is the input string double qouted ('"input"')? What does the backslash annotate? At first it looks like an escape similar to regex, but the other bracket doesn't have it so it must mean something else? Etc. Of course I could look the answers up, but the point is I can't tell without looking it up. While the code is short it feels similar to Awk, if you know it it's trivial, if you don't know it it's hard to be sure what it does without trying it out.
jq seems shorter but I don't see how it's more intuitive - ".name" is pretty basic jq syntax but how do I know \() is syntax for wrapping selectors within strings? I've used jq for many years and tbh I've never done string templating with it, so I would need to go digging in the docs.
You can argue that's the case for any language, but it's the fundamental reason DSLs rarely capture the mainstream for on-the-fly utilities. Leveraging existing knowledge of a more heavily used syntax really helps here. Look at awk for example - it's been around forever, and despite its power and ubiquity in OS installs, you'll still see sed in many many more bash scripts because it's leaning on widely used familiar syntax rather than a DSL.
That said - the fx example has "x => Hello, ${x}!" where I would expect "x => `Hello, ${x}!`", so that does indicate to me that fx has its own gotchas lurking - this example could just be contrived to look intuitive.
After looking more at fx that seems to be the case yes! It's 100% pure js syntax so shouldn't have any gotchas.
jq is YADSL though vs JS which is ubiquitous. Same reason most people can't use awk or sed.
I know JavaScript, so I immediately know a lot of the `fx` syntax.
I always found the `jq` syntax required many round-trips to the mail and more trial and error to get the syntax right.
I know JavaScript, so I immediately know a lot of the `fx` syntax.
I always found the `jqc
jq is probably better if you don't have node or deno installed since fx can't do expressions without it
Happy jq user as well.
But is jq interactive?
ijq [1] is. I am using it almost every time I need to come up with a jq expression. However, it gets noticeably laggy with enormous JSONs.
Very cool. Not a huuge deal, but for open source software, I don't totally appreciate the http call to a bitly tracker in the install script...
Just use:
go install github.com/antonmedv/fx@latestPretty cool! I actually wrote something VERY similar a couple of years ago: sless[1]. It's a tool for viewing json-based structured logs. Just like your tool, you can explore into a json object. The difference is, it expects the input to have many json objects, newline separated, and it shows few keys as a preview of the object, to make looking for something in the log easier. It's not quite complete but basic browsing works. It was mainly written to learn more about Urwid[2], a library similar to Curses.
The other day, I found myself in need of visualising a tree of paths, like this:
/one/two
/one/three/four
/one/three/five
/six
(Incidentally, I’ve been writing my own viewer [0] to satisfy my original need more straightforwardly, but that’s still in very early stages.)
As in filesystem paths? Are you aware of `tree`?
Not necessarily filesystem paths (in my case, the paths correspond to RocksDB keys). And I’d like to be able to fold/unfold subtrees, search within node names, and call arbitrary commands for a given path in the tree, from within a TUI. So no, `tree` won’t cut it.
This could be done with nushell.
sure can do, if you already use that shell [1], but personally I like specific tools for specific jobs such as jq [2], fx, csvq [3] etc, there's value in decoupling shells from utils (modularity, speed, innovation etc).
[1] I don't but tempted to try, like its data-types concept
Is the output ascii diagram only? Seems like a pretty different use case
No the output is a table object, you can apply other operations to it like sort, etc. the shell displays them in this ascii art way at the end but you can also write to files in various formats.
Any comparison to jless?
fx looks to work from the command line (without viewing) to do more than just inspect, looks like you can do pretty much anything to it with a bit of JS, though not from the TUI afaict
There are many JSON viewers, I would think that inspecting with collapsable sections ought to be right in VS Code with LSP/treesitter, no extra tools needed.
What's more interesting to me is what can you do with the JSON in these tools, validate, filter, transform, chain...?
I find `gron` to be very helpful for generating jq query strings
> Built using the Go programming language for efficient performance.
Do you mean to say "efficiency" or "performance"? I dont see both working like that
It's correct as it is. Performance can be good or bad. "...for efficiency" would be correct, but if they were to write only "...for performance" it would missing a qualifier.
FYI: .wtf top domain is blocked in HaGeZi's The World's Most Abused TLDs: https://raw.githubusercontent.com/hagezi/dns-blocklists/main...
Might wanna change to something else.
This is ridiculous. I will not change my domain because of someone’s blocklist.
Used to use HaGeZi's Blocklist but kept hitting false positives like this.
Need to have a bit more of a play with it but I really like it so far. I mainly see myself using it to pipe into (like from curl or the gh cli).
One thing that isn't clear to me is how to actually use the themes you can display with fx --themes.
Will update the docs. But it is easy:
FX_THEME=2 fx data.jsonSomething that would be extra awesome would be light markdown support inside fields. This would make it useful for reviewing chatbot conversations.
Looks great!
What is the website https://fx.wtf/ built-in?
Vue
it is written in Go. What does it have to do with npx/npm at all? Did I miss something about these tools?
Where do you see npm/npx mentioned?
The project supports JS scripts as arguments to process the data:
https://fx.wtf/getting-started#json-processing
It's hard to tell from the docs if there is any JS involved if you just pipe JSON data to fx. I would hope not, since it mentions being written in Go for performance.
This is what I was wondering as well. Is there an interpreter built into the go binary? or is it using my node install under the hood?
Edit: Answered my own question. I installed it into a podman container with nothing else in it, and it printed this when trying to execute js:
Node.js or Deno is required to run fx with reducers.strace (on Linux) would have told you that without needing to user podman
at https://fx.wtf/getting-started it says:
$ cat file.json | npx fx .field
It seems it would work since they release an npm package too: https://www.npmjs.com/package/fx
A bit surprising, I wonder if they implemented it in JS too, or somehow managed to distribute their Go program on npm.
Edit: They indeed have a separate non-interactive version in JS: https://github.com/antonmedv/fx/tree/master/npm
It is nice for PCs, but requiring Node.js limits its use on servers.
This looks really nice.
visidata anyone?
This fx rewrite is very exciting. I'll have to try it. I thought of fx as a wrapper around jq, that allowed quick iteration over building jq scripts. Sort of an Ultimate Plumber [1] but only for jq. It looks like it is now more like a JavaScript processor plus an interactive viewer.
Someone mention Visidata[2]? VisiData is also a TUI that is great on tabular data, and it can work with json. If your JSON is mostly tabular in nature, Visidata does a great job at showing that data and allowing you to explore it. A lot of json I deal with is tabular-like data. There is a great tutorial [3], that can help you get your bearings with Visidata. Once you understand those basics you might want to look at this thread [4] for what commands you can use with json.
[1] Ultimate Plumber: https://github.com/akavel/up
[2] Visidata: https://github.com/saulpw/visidata
[3] "Intro to VisiData Tutorial": https://jsvine.github.io/intro-to-visidata/ by Jeremy Singer-Vine
[4] "Is it possible to "flatten" structured data (like JSON?)": https://github.com/saulpw/visidata/discussions/1605
amazing! Does someone know something like this for yaml? (does not have to be cli)
could've used this a few years ago before jq became so second nature, cool
What can you do with the JSON in the TUI? Anything besides looking at it? (https://fx.wtf/getting-started#json-processing but from the TUI?)
Built something similar, but with love for all things CUE (also works for JSON/Yaml) (https://youtu.be/XNBqBWO4y08)
Yours looks to have a better object navigation, I might have to be inspired by it!
How do you like Bubbletea? We went with tcell/tview (fork) for more control of the UI & event handling
wow nice, I'll give this a try.
I've always used jq for viewing json, but it's the wrong tool for my use. I basically want to explore json interactively, not parse it specifically. So the workflow is to keep "jq"ing it, and modifying my jq query until I find what I'm looking for.
Your tool looks like it allows interactive exploration.
You may be interested in an interactive jq tool like jiq or others: https://github.com/fiatjaf/awesome-jq (do a ctrl-f for "interactive" on this page).
That page doesn't include jless[1], which seems very similar to fx and will give you a jq selector for whatever node you're on
1: https://jless.io/
This thread is a gem after gem, thank you!
Some feedback:
- Some indication of how far down you've scrolled would be useful for "situational awareness". Perhaps a percentage number in the status bar (like emacs/vim?)
- I miss not having the Home/End keys bound to the goto top/bottom actions
- The cursor behavior when pressing PageDn/PageUp throws me off.
Emacs when pressing PageDn: Cursor ends up at the top of the window.
Vim: Cursor stays at the current position.
notepad.exe: Cursor ends up in the middle of the window.
Fx: Cursor ends up at the bottom of the window.
- The . navigation is very neat with the tab completion. Probably stupid observation: If you remove the alphabet key shortcuts you wouldn't need the (non-discoverable) dot. :)
Thanks for the feedback. Will try to address it in future releases.
Yes pgdown behaviour need to be refactored. Home/end is easy to add.
> If you remove the alphabet key shortcuts you wouldn't need the (non-discoverable) dot. :)
I did not get this one.
This looks really nice, well done!
I never could get used to jq's syntax so instead I've just been grepping. But this looks to be for jq what htop is to top, which is great for exploring response data.
Hello,
Is there a commonly-used grammar for "json with comments and trailing commas" or does everyone just try to copy the jsonc from vscode?
It's basically https://json5.org/
CUE will happily take your JSON with comments and extra commas, or no commas too (lists still need them), most keys can be unquoted, and it's just a one-liner to get valid json or yaml from it
Jsonnet does too I believe. I wanted to like Cue but it only works with Go and it doesn't actually let you link the schemas you write from the "document" files, which means no automatic schema checking and no IDE support. Giving up two huge benefits of having a schema in the first place.
In an XML document you can specify the XML schema location with `xsi:schemaLocation`. In a JSON document you can specify the JSON Schema location with `$schema`.
In a Cue document how do you specify the location of the Cue schema? As far as I can tell you can't.
And yes I am aware that the concept of a "document" and a "schema" are basically merged in Cue, but that doesn't mean I don't want to do this.
> CUE is most often used as a CLI, so no Go is needed
Only because that's the only way you can use it.
> Being Go, you can compile to SO or WASM
Compiling to a shared library is awkward because you then need to have the Go toolchain installed. WASM might be an option ... I couldn't find out how Go actually works in WASM though given that it needs a GC and threads, and WASM currently has neither. Do they ship their own WASM GC?
Either way, it's of no practical use since nobody has actually done that. Look at the languages supported by Jsonnet for comparison: https://jsonnet.org/ref/bindings.html
Python, C, Rust (twice), Go (twice), Lua, Node, PHP, Ruby, Haskell.
If I'm loading a config file from Python I don't want to have to mess around with `subprocess.run()` to read it. It's a solvable problem... but today, it's a bit of a turn-off.
http://hyperrealm.github.io/libconfig/ ?
This looks like an implementation of some totally different format. Additionally, the authors of that format chose not to specify some parts of its grammar.
I am probably missing something but is interactive mode not a flag? Is it just what happens with a longer JSON automatically?
how do you use the Built-in Functions? i tried them but it gives me error every time
Why not call it Jim? JSON vim