Add To/FromFormMultiPart classes #24

Activity

1. fizruk commented on Feb 19, 2018

   

   fizruk

   on Feb 19, 2018

   Last edited by fizruk

   OwnerAuthor

   More actions

   Related work

   Server-side:

   • servant-multipart implements server-side code for handling multipart/form-data;
   • yesod-form has support for file uploads (see Cookbook file upload saving files to server).

   Client-side:

   • experimental work in Support multipart/form-data encoding in servant-client haskell-servant/servant#884 (comment) to support multipart/form-data for servant-client;
   • http-client's Network.HTTP.Client.MultipartFormData;
   • custom multipart/form-data implementation in telegram-api's Servant.Client.MultipartFormData.

   EDIT: added yesod-form.
2. fizruk

   closed this as completedon Feb 21, 2018
3. fizruk

   reopened this on Feb 21, 2018
4. fizruk commented on Feb 21, 2018

   

   fizruk

   on Feb 21, 2018

   OwnerAuthor

   More actions

   yesod-form also has support for file upload (see Cookbook file upload saving files to server).
   File upload field is created using fileAFormReq. File info is represented with FileInfo:

   data FileInfo = FileInfo
       { fileName        :: !Text
       , fileContentType :: !Text
       , fileSourceRaw   :: !(ConduitT () ByteString (ResourceT IO) ())
       , fileMove        :: !(FilePath -> IO ())
       }

   They also seem to have 3 options for handling file uploads (on the server):

   data FileUpload
     = FileUploadMemory !(NWP.BackEnd L.ByteString)
     | FileUploadDisk !(InternalState -> NWP.BackEnd FilePath)
     | FileUploadSource !(NWP.BackEnd (ConduitT () ByteString (ResourceT IO) ()))

   Note the last option which is not available with servant-multipart AFAICT.
5. phadej commented on Feb 21, 2018

   

   phadej

   on Feb 21, 2018 via email

   Collaborator

   More actions

   It isn't, but iirc can be added in an app: implement MultipartBackend

   …

   Sent from my iPhone

   On 21 Feb 2018, at 17.05, Nickolay Kudasov ***@***.***> wrote: yesod-form also has support for file upload (see Cookbook file upload saving files to server). File upload field is created using fileAFormReq. File info is represented with FileInfo: data FileInfo = FileInfo { fileName :: !Text , fileContentType :: !Text , fileSourceRaw :: !(ConduitT () ByteString (ResourceT IO) ()) , fileMove :: !(FilePath -> IO ()) } They also seem to have 3 options for handling file uploads (on the server): data FileUpload = FileUploadMemory !(NWP.BackEnd L.ByteString) | FileUploadDisk !(InternalState -> NWP.BackEnd FilePath) | FileUploadSource !(NWP.BackEnd (ConduitT () ByteString (ResourceT IO) ())) Note the last option which is not available with servant-multipart AFAICT. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.
6. fizruk commented on Feb 21, 2018

   

   fizruk

   on Feb 21, 2018

   OwnerAuthor

   More actions

   The more I think about it the more it seems like we should just provide common data types and not use type classes 🙃

   For instance we could just provide something like this:

   data MultiPartFormData file = MultiPartFormData
     { mpfdInputs :: Form
     , mpfdFiles :: [FileData file]
     }
   
   data FileData file = FileData
     { fdInputName :: Text  -- input field name
     , fdFileName :: Text  -- user's file name
     , fdFileHeaders :: [Header] -- additional file headers
     , fdContentType :: Text
     , fdFileContents :: file
     }
   
   -- | Locally saved file.
   newtype LocalFilePath = LocalFilePath FilePath
   
   -- | File contents loaded into memory.
   newtype FileContents = FileContents ByteString
   
   -- | Some file that can be read directly or moved (saved) somewhere.
   -- Particularly useful for server-side handlers to ignore how server handles uploads.
   data SomeFile = SomeFile
     { someFileSorceRaw :: ConduitT () ByteString (ResourceT IO) () -- I don't really know how to convert this into a simpler type
     , someFileMove :: FilePath -> IO ()
     }

   Now we could provide classes like

   class ToMultiPartFormData file a where
     toMultiPartFormData :: a -> MultiPartFormData file
   
   class FromMultiPartFormData file a where
     parseMultiPartFormData :: MultiPartFormData file -> Either Text a

   However those are multi parameter and without functional dependencies so are likely to only cause confusion.

   Alternatively we could limit file for client/server:

   class ToMultiPartFormData a where
     toMultiPartFormData :: a -> MultiPartFormData (IO ByteString)
   
   class FromMultiPartFormData file a where
     parseMultiPartFormData :: MultiPartFormData SomeFile -> Either Text a

   However those look pretty weird to me, especially with IO actions in IO ByteString and SomeFile.

   So for now I think it would be sufficient to just have common datatypes and functions and not introduce type classes for now.

   @alpmestan @phadej do you have any thoughts on this?
7. alpmestan commented on Feb 21, 2018

   

   alpmestan

   on Feb 21, 2018

   More actions

   The more I think about it the more it seems like we should just provide common data types and not use type classes 🙃

   Surely you'll reconsider this once your head is not upside down anymore. =)

   More seriously: I do think typeclasses are useful here, as with captures, query params and what not. The handlers are then allowed to only care about the user's "business types" (e.g data User = User { name :: String, profilePic :: FilePath }, where the picture is uploaded with our machinery and the name sent along in the form). I won't fight people to death on this, of course not, but it would be consistent with everything else. Also, do the From/ToMultipart classes have to be parametrized by what's effectively the backend (file) ? Could we not have:

   class ToMultiPartFormData a where
     toMultiPartFormData :: forall backend. KnownBackend backend => a -> MultiPartFormData backend
   
   class FromMultiPartFormData a where
     parseMultiPartFormData :: forall backend. KnownBackend backend => MultiPartFormData backend -> Either Text a

   where KnownBackend would be something similar to what we have in servant-multipart ? Again, I'm not too attached to these suggestions, feel free to dismiss them. The most important thing here for me is to have a story for migrating servant-multipart over to whatever we end up settling on by the time it is ready. :)
8. fizruk commented on Feb 21, 2018

   

   fizruk

   on Feb 21, 2018

   Last edited by fizruk

   OwnerAuthor

   More actions

   @alpmestan I think the problem with KnownBackend is that you can't easily write

   data User = User
     { name :: String
     , profilePic :: FilePath
     }
   
   instance FromMultiPartFormData User where
     parseMultiPartFormData :: KnownBackend backend =>
       MultiPartFormData backend -> Either Text User
     parseMultiPartFormData = ... -- what goes here?

   At least I can't figure out how to do that with servant-multipart's class:

   class MultipartBackend tag where
       type MultipartResult tag :: *
       type MultipartBackendOptions tag :: *
   
       backend :: Proxy tag
               -> MultipartBackendOptions tag
               -> InternalState
               -> ignored1
               -> ignored2
               -> IO SBS.ByteString
               -> IO (MultipartResult tag)
   
       defaultBackendOptions :: Proxy tag -> MultipartBackendOptions tag

   If I understand it correctly, MultipartBackend instances specify how to obtain MultipartResult tag values. However you can't really use backend to implement a backend-agnostic FromMultipart instance (because you have to know how to break down MultipartResult tag, but you only know how to make one with backend).
9. alpmestan commented on Feb 21, 2018

   

   alpmestan

   on Feb 21, 2018

   More actions

   Hmm, correct, nevermind my suggestion then!
10. fizruk

    added

    help wanted

    on Sep 24, 2018