• How to Display Custom Images in Tensorboard

    How to Display Custom Images in Tensorboard

    The Image Dashboard section of the Tensorboard ReadMe says:
    Since the image dashboard supports arbitrary pngs, you can use this to embed custom visualizations (e.g. matplotlib scatterplots) into TensorBoard.
    I see how a pyplot image could be written to file, read back in as a tensor, and then used with tf.image_summary() to write it to TensorBoard, but this statement from the readme suggests there is a more direct way. Is there? If so, is there any further documentation and/or examples of how to do this efficiently?
    It is quite easy to do if you have the image in a memory buffer. Below, I show an example, where a pyplot is saved to a buffer and then converted to a TF image representation which is then sent to an image summary.


    import io
    import matplotlib.pyplot as plt
    import tensorflow as tf
    
    
    def gen_plot():
        """Create a pyplot plot and save to buffer."""
        plt.figure()
        plt.plot([1, 2])
        plt.title("test")
        buf = io.BytesIO()
        plt.savefig(buf, format='png')
        buf.seek(0)
        return buf
    
    
    # Prepare the plot
    plot_buf = gen_plot()
    
    # Convert PNG buffer to TF image
    image = tf.image.decode_png(plot_buf.getvalue(), channels=4)
    
    # Add the batch dimension
    image = tf.expand_dims(image, 0)
    
    # Add image summary
    summary_op = tf.summary.image("plot", image)
    
    # Session
    with tf.Session() as sess:
        # Run
        summary = sess.run(summary_op)
        # Write summary
        writer = tf.train.SummaryWriter('./logs')
        writer.add_summary(summary)
        writer.close()
    This gives the following TensorBoard visualization:
    enter image description here
    A bit late with my answer. With tf-matplotlib a simple scatter plot boils down to:
    import tensorflow as tf
    import numpy as np
    
    import tfmpl
    
    @tfmpl.figure_tensor
    def draw_scatter(scaled, colors): 
        '''Draw scatter plots. One for each color.'''  
        figs = tfmpl.create_figures(len(colors), figsize=(4,4))
        for idx, f in enumerate(figs):
            ax = f.add_subplot(111)
            ax.axis('off')
            ax.scatter(scaled[:, 0], scaled[:, 1], c=colors[idx])
            f.tight_layout()
    
        return figs
    
    with tf.Session(graph=tf.Graph()) as sess:
    
        # A point cloud that can be scaled by the user
        points = tf.constant(
            np.random.normal(loc=0.0, scale=1.0, size=(100, 2)).astype(np.float32)
        )
        scale = tf.placeholder(tf.float32)        
        scaled = points*scale
    
        # Note, `scaled` above is a tensor. Its being passed `draw_scatter` below. 
        # However, when `draw_scatter` is invoked, the tensor will be evaluated and a
        # numpy array representing its content is provided.   
        image_tensor = draw_scatter(scaled, ['r', 'g'])
        image_summary = tf.summary.image('scatter', image_tensor)      
        all_summaries = tf.summary.merge_all() 
    
        writer = tf.summary.FileWriter('log', sess.graph)
        summary = sess.run(all_summaries, feed_dict={scale: 2.})
        writer.add_summary(summary, global_step=0)
    When executed, this results in the following plot inside Tensorboard
    
    
    
    
    
    
    
    
    Next script does not use intermediate RGB/PNG encoding. It also fixes the issue with additional operation construction during execution, single summary is reused.
    Size of the figure is expected to remain the same during execution
    Solution that works:
    import matplotlib.pyplot as plt
    import tensorflow as tf
    import numpy as np
    
    def get_figure():
      fig = plt.figure(num=0, figsize=(6, 4), dpi=300)
      fig.clf()
      return fig
    
    
    def fig2rgb_array(fig, expand=True):
      fig.canvas.draw()
      buf = fig.canvas.tostring_rgb()
      ncols, nrows = fig.canvas.get_width_height()
      shape = (nrows, ncols, 3) if not expand else (1, nrows, ncols, 3)
      return np.fromstring(buf, dtype=np.uint8).reshape(shape)
    
    
    def figure_to_summary(fig):
      image = fig2rgb_array(fig)
      summary_writer.add_summary(
        vis_summary.eval(feed_dict={vis_placeholder: image}))
    
    
    if __name__ == '__main__':
          # construct graph
          x = tf.Variable(initial_value=tf.random_uniform((2, 10)))
          inc = x.assign(x + 1)
    
          # construct summary
          fig = get_figure()
          vis_placeholder = tf.placeholder(tf.uint8, fig2rgb_array(fig).shape)
          vis_summary = tf.summary.image('custom', vis_placeholder)
    
          with tf.Session() as sess:
            tf.global_variables_initializer().run()
            summary_writer = tf.summary.FileWriter('./tmp', sess.graph)
    
            for i in range(100):
              # execute step
              _, values = sess.run([inc, x])
              # draw on the plot
              fig = get_figure()
              plt.subplot('111').scatter(values[0], values[1])
              # save the summary
              figure_to_summary(fig)
    This intends to complete Andrzej Pronobis' answer. Following closely his nice post, I set up this minimal working example:
        plt.figure()
        plt.plot([1, 2])
        plt.title("test")
        buf = io.BytesIO()
        plt.savefig(buf, format='png')
        buf.seek(0)
        image = tf.image.decode_png(buf.getvalue(), channels=4)
        image = tf.expand_dims(image, 0)
        summary = tf.summary.image("test", image, max_outputs=1)
        writer.add_summary(summary, step)
    Where writer is an instance of tf.summary.FileWriter. This gave me the following error: AttributeError: 'Tensor' object has no attribute 'value' For which this github post had the solution: the summary has to be evaluated (converted into a string) before being added to the writer. So the working code for me remained as follows (simply add the .eval() call in the last line):
        plt.figure()
        plt.plot([1, 2])
        plt.title("test")
        buf = io.BytesIO()
        plt.savefig(buf, format='png')
        buf.seek(0)
        image = tf.image.decode_png(buf.getvalue(), channels=4)
        image = tf.expand_dims(image, 0)
        summary = tf.summary.image("test", image, max_outputs=1)
        writer.add_summary(summary.eval(), step)
    This could be short enough to be a comment on his answer, but these can be easily overlooked (and I may be doing something else different too), so here it is, hope it helps!


  • 0 comments:

    Post a Comment

    New Research

    Attention Mechanism Based Multi Feature Fusion Forest for Hyperspectral Image Classification.

    CBS-GAN: A Band Selection Based Generative Adversarial Net for Hyperspectral Sample Generation.

    Multi-feature Fusion based Deep Forest for Hyperspectral Image Classification.

    ADDRESS

    388 Lumo Rd, Hongshan, Wuhan, Hubei, China

    EMAIL

    contact-m.zamanb@yahoo.com
    mostofa.zaman@cug.edu.cn

    TELEPHONE

    #
    #

    MOBILE

    +8615527370302,
    +8807171546477