is there any reason no session or constant value graph after execution not available in python land #108

Activity

1. yupbank

   changed the title [-]is there any reason no session or constant value graph after execution not available in python lab[/-] [+]is there any reason no session or constant value graph after execution not available in python land[/+] on May 12, 2017
2. thunterdb commented on May 16, 2017

   

   thunterdb

   on May 16, 2017

   Contributor

   More actions

   Yes, the execution session is fully managed on the workers and you do not have access to the sessions. This has a number of benefits such as allowing the sessions to be transparently multiplexed.

   I have not tried these solutions yet, but if you want to collect some variables from the workers, there are a couple of ways, I believe:

   • for asynchronous communication, you should be able to use the primitives from tensorflow. The only issue concerns retrieving the final batch, but in most algorithms, the last batch is not so important
   • for synchronous communication, you should be able use tfs.reduce, once I add the session caching. You run a first map_rows phase to fill the variables, and then in a second phase, you merge these variables using reduce.

   The second option requires a few incremental changes on the scala side to work though. I am a bit busy right now, but I should be able to take a look at it in three weeks - unless you want to take a look first.
3. yupbank commented on May 16, 2017

   

   yupbank

   on May 16, 2017

   ContributorAuthor

   More actions

   cool cool, I'll look at it when i got time..

   btw, is there a google forum or something so we can communicate more freely
4. yupbank commented on May 26, 2017

   

   yupbank

   on May 26, 2017

   Last edited by yupbank

   ContributorAuthor

   More actions

   Hi, @thunterdb so can you give me some hint on use the primitives from tensorflow for retrieving data from running graph? I'm looking into it but doesn't get any clue