Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prefer web APIs over node APIs #13

Open
Gozala opened this issue Nov 14, 2019 · 3 comments
Open

Prefer web APIs over node APIs #13

Gozala opened this issue Nov 14, 2019 · 3 comments
Assignees
Labels
enhancement New feature or request

Comments

@Gozala
Copy link
Collaborator

Gozala commented Nov 14, 2019

I often struggle using JS-IPFS in the browser because it predates all the nice web APIs that made working with binary data and files convenient. Often time this implies to using some node polyfills and / or needing to convert node constructs to browser ones so that other browser APIs can interact with it.

Here are few things that I have noticed.

  • addFile depends on node read stream etc..
    https://github.com/textileio/js-ipfs-lite/blob/0bb38fc91e10c480aef3109fde9ef98612a9c5a4/src/index.ts#L110-L124

    I would suggest to use web native File API instead which can have a path and can be constructed in arbitrary ways.

    As of files bundles there is FormData and I think it would make more sense to us that instead. It also has advantage that web forms can directly be written to IPFS. And all this works well with fetch and XMLHTTPRequest stuff.

  • getFile returns node Buffer. Which luckily in browser pollyfill implemented as a subclass of Uint8Array so it is somewhat more usable. However that still implies allocating memory for the whole content etc... On the other hand web has a Response API available who's body can be read as string, bytes, json, FormData or a stream. In fact it's also possible to use slices over the whole thing and also has notion of Error. It also can be stored in browser cache etc...

I would propose to draw inspiration from web native APIs and try to be as similar to fetch API as possible so it can be used as a drop in replacement. Specifically I'd very much prefer following intterface:

interface Peer {
  put(FormData|File|Blob):Promise<CID>
  get(CID|Path):Response;
  head(CID|Path):Response; // 404 if not found, 502 if network is spotty
  delete(CID):Response; // Delete from local storage
}
@carsonfarmer
Copy link
Collaborator

carsonfarmer commented Nov 14, 2019

Thanks! These are great suggestions.

We definitely have plans to expose browser native APIs via both getFile and addFile, so your suggestions will help define what that will look like :)

The one catch at the moment is, we need the IPFS-Lite interface to match that of js-ipld, or at least, those core methods for now. So that's why we have the getFile and addFile as separate APIs in there (they aren't part of the IPLD interface).

Having said that, as is, addFile does not depend on node read streams, that's just an example. At the moment we are depending on https://github.com/ipfs/js-ipfs-unixfs-importer{/exporter}, which does indeed use Buffers and other less-nice things. This was mostly to avoid having to re-write everything from scratch... but I'd like to re-implement some of that anyway, as it is actually pretty heavy handed for what we need here for the most part. Good call.

I really like the Response suggestion for getFile... let's do that. Though not sure sure what your head API would be used for?

Similarly, addFile most certainly should support FormData and the native Files/Blob API. I'll create separate issues for those two (and certainly, PRs are welcome :)).

@carsonfarmer
Copy link
Collaborator

carsonfarmer commented Nov 14, 2019

Building on this, I think I have a nice idea to make the API easier to use... see https://github.com/textileio/js-ipfs-lite/tree/carson/restructure/src/files which is where we could put all the nicer browser native files stuff... an example of how we'd do this in a browser/nodejs compatible way is here: https://github.com/textileio/js-ipfs-lite/tree/carson/restructure/src/setup

@carsonfarmer carsonfarmer self-assigned this Nov 20, 2019
@carsonfarmer carsonfarmer added the enhancement New feature or request label Nov 20, 2019
@carsonfarmer carsonfarmer added this to the Sprint 25 milestone Nov 20, 2019
@rumkin
Copy link

rumkin commented Jan 31, 2020

I'm working on web server which is based on WebAPI Streams, and there is one issue with the disturbed stream. You can read more here: whatwg/streams#1025. In a few words, there is no way to specify wether the current stream will provide an initial chunk of data on read in a browser. But there is a hack to do so:

function isStreamDisturbed(stream: ReadableStream): Boolean {
  try {   
      new Response(stream)
    
      return false
    }
    catch {
      return true
    }
}

const stream = new ReadableStream()
stream.cancel()

isStreamDisturbed(stream) // -> true

@carsonfarmer carsonfarmer removed this from the Sprint 25 milestone Feb 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy