3 Replies Latest reply on May 24, 2017 10:27 PM by pawans

    APIs for getting a data dump - can we restrict access?



      We've started on a mission to clean up, tag and categorise contents on one of our community's most active group that has around 6.5k pieces of content. We want to make the content more searchable by adding meta data to the contents. We're doing this project as a crowd sourced one. Any help/advice/suggestion on the following questions is welcome


      1. We need a dump of the entire data that's on the group currently. If we had to give people (those who volunteered for the cleanup) JIVE API access for getting all the contents, what's the best way to do this without running the risk of exposing any non-public content on the platform? Is there a way to limit a certain API's access? Will Jive's OAuth be of any help here? Any documentation around this?

      2. What's the best way to figure out the best tags and categories to be assigned to the contents, given the huge numbers that we're dealing with? To what level can we utilise machine learning ML for this activity?

      3. If we had to keep this as an ongoing activity, how do you suggest we keep tagging/categorising content as they keep coming into Jive?



        • Re: APIs for getting a data dump - can we restrict access?
          Ryan Rutan

          Your request is a bit broad, but I'll do my best to address some of the key points here.


          1. Giving API access to people is relatively straight forward, if an SSO environment, you would need to create a means for them to do a 3-legged OAuth dance to self-service an OAuth token. Lots of examples of doing this with a simple App, or even a Tile.
            • Most* OAuth Tokens are granted with an /api/* scope, which means they can call any API method, but the data it returns and actions it allows use managed permissions under the hood.
            • Keeping the OAuth Token safe is your biggest issue.  If the token gets compromised ...you can revoke access through the User Authorizations UI.
            • If you are truly trying to crowd-source this..I would create some proxy mechanism to broker the tokens and pair it up with an existing enterprise SSO (or some other identifying measure) ... such that the actual token is never revealed to the user.  This will greatly reduce your risk, however, since you are crowd sourcing and giving people direct API access..you have to expect that people will make mistakes.  I would strongly recommend not relying on people to call the API directly, but rather give them contextually relevant experiences that simplify the work to chip away at the work to be done.  Crowd-sourcing works better when the unit of work is small and simple, rather than large and complex.
          2. I think you answered your own question here =)  If you are looking to get some insights from all the content in a group, running the text through a machine learning process would be a great way to glean these insights.  From there, you can make plans accordingly.
          3. It all depends on what you are trying to build.  For example, if you wanted to make this a program for "zealots" or "trusted community users", here is what I would do to make it a bit more formal and long-lasting
            • Build a middleware service that uses DES or Webhooks to keep up to date on all content contributed to specific places in Jive.
            • Middleware service runs content through your ML process (as you suggested), and you can maintain a running index of terms and scores per place.
            • Front this ML term index with a service that can be invoked via REST/JSON by place
            • Build a Jive App (for authorized users only, i.e. internal/zealots, etc...) that allows users to take quick action from either the place-level or content-level (or Global-level)
            • Inside the App, you do not need to manage OAuth tokens...as you can effectively use the Jive app session with/out Jive Connects


          That's about all I have off the top of my head, but basically ... it sounds like you are wanting to build a formal program/project around this endeavor....


          I know that Pawan Shah has a lot of experience with stuff like this for his company, so I'll bring him in to see if he has any further thoughts.

          See: Content Manager App - The idea into reality


          Hope this helps get you started.

          4 people found this helpful