-
Notifications
You must be signed in to change notification settings - Fork 328
Streaming CLI Output #608
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Streaming CLI Output #608
Conversation
cec92e9
to
bde5f83
Compare
@cduchesne Would you please test this when you get a chance? Thanks! |
Codecov Report
@@ Coverage Diff @@
## master #608 +/- ##
==========================================
- Coverage 34.16% 8.96% -25.21%
==========================================
Files 36 15 -21
Lines 2901 2299 -602
==========================================
- Hits 991 206 -785
- Misses 1807 2068 +261
+ Partials 103 25 -78
Continue to review full report at Codecov.
|
bde5f83
to
5577d3f
Compare
This patch adds a new feature to REX-Ray's Volume CLI so that streaming output is supported. By setting the environment variable "REXRAY_CLI_TABWRITER_AUTOFLUSH" to a truthy value, the CLI will emit results as they are produced rather than waiting until all results are available. For example: $ REXRAY_CLI_TABWRITER_AUTOFLUSH=true \ rexray volume create build1 build2 build3 --size 16 The above command will create three volumes named "build1", "build2", and "build3" with a size of 16GB. With the above environment variable set, however, the CLI will emit the volumes as they're created instead of waiting until all three have been processed. If an error occurs when processing multiple items, the error is written to the console in the middle of the normal, tabular output. However, the error is written to the STDERR file descriptor instead of STDOUT, meaning it is very simple to pay attention to only the successful results if so desired. The streaming output is also supported for the "json" and "jsonp" formats. There is one caveat to using the streaming output option -- there is no longer a guarantee that tabular data will be properly aligned. While the data will still be separated by tabs, the data must normally be buffered until it is all available to know how to properly produce the formatted table. Streaming the data means padding can only be generated based upon the current and previous entries, but not possible, future ones. For example: ID Name Status Size Path vol-0 builds-0 available 10240 vol-1 builds-1 available 10240 error querying volume vol-2 vol-3 builds-3-abcdefghik available 10240 vol-4 builds-4 available 10240 The above output demonstrates two things: 1) an error emitted to STDERR in the middle of the tabular data that has been written to STDOUT and 2) the entry for "vol-3" is not properly aligned due to the fact that the tabular formatting was calculated well in advance of the "vol-3" data being taken into consideration.
Hi @cduchesne, Do you have some time to please verify this? Thank you. |
Hi @cduchesne, Do you have some time to please verify this? Thank you. I think like the image above, @cduchesne may have gone off the deep-end with meetings. Can you possibly validate this PR? Thanks! |
Hi @akutz I grabbed your branch and built it, to test on CentOS7 with vbox driver. Operations like The bigger issue I ran into, though, was that I couldn't create Volumes in vbox with rexray built from this branch, but I could from master. The debug output is large, so I put it in a gist: https://gist.github.com/codenrhoden/5f0246ea68cccd58d651efc5ea60865c The test version hangs after 'created libstorage client' -- the master version proceeds, and starts to use the executor. I do not that this PR needs to be rebased, so it may not be changes in this PR at all, but wanted to bring it up. Same config for both setups: libstorage:
service: virtualbox
virtualbox:
endpoint: http://10.0.2.2:18083
volumePath: /Users/travis/VirtualBox VMs/Volumes
controllerName: IDE Controller |
This patch adds a new feature to REX-Ray's Volume CLI so that streaming output is supported. By setting the environment variable
REXRAY_CLI_TABWRITER_AUTOFLUSH
to a truthy value, the CLI will emit results as they are produced rather than waiting until all results are available.For comparison, here is what traditional, buffered output looks like:
When enabled, streaming output will appear as results are produced:
To better understand streaming output, please consider the following example:
The above command will create three volumes named
build1
,build2
, andbuild3
with a size of 16GB. With the above environment variable set, however, the CLI will emit the volumes as they're created instead of waiting until all three have been processed.If an error occurs when processing multiple items, the error is written to the console in the middle of the normal, tabular output. However, the error is written to the
STDERR
file descriptor instead ofSTDOUT
, meaning it is very simple to pay attention to only the successful results if so desired.The streaming output is also supported for the
json
andjsonp
formats.There is one caveat to using the streaming output option -- there is no longer a guarantee that tabular data will be properly aligned. While the data will still be separated by tabs, the data must normally be buffered until it is all available to know how to properly produce the formatted table. Streaming the data means padding can only be generated based upon the current and previous entries, but not possible, future ones. For example:
The above output demonstrates two things: 1) an error emitted to
STDERR
in the middle of the tabular data that has been written toSTDOUT
and 2) the entry forvol-3
is not properly aligned due to the fact that the tabular formatting was calculated well in advance of thevol-3
databeing taken into consideration.