Hub
Pricing About
WorkflowWorkflow

01_caption_preprocessing

Image processingImage analysisImage captioningDeep learningNeural networks
+3
bwilhelm profile image
Draft Latest edits on 
Mar 31, 2020 8:59 AM
Drag & drop
Like
Download workflow
Workflow preview
After we cleaned the training captions and pre-calculated image-/word- features, the caption network can be trained. In this example, the task of image captioning is modelled as an iterative approach predicting the captions word-by-word. For this, we input the network with an image, a partial caption, and let it predict the next word in the caption. This also means we model the task as a word classification (using our vocabulary as possible classes). Before we can start training, we have to bring the data into this iterative format. For each image/caption pair, we create several training examples using all possible partial sentences until all words in the caption have been used as target word once. We use a simple network with two input branches for training. The first branch contains some dense layers to further process the image feature vector. The second branch contains an embedding layer to map our encoded caption to GLOVE vectors. This is achieved by setting the parameters of the embedding vector to the previously created Python dictionary and making the layer untrainable. As output, the workflow writes the trained model to disk.

External resources

  • COCO Dataset Homepage
  • COCO 2014 Data Download
  • Image Captioning
Loading deploymentsLoading ad hoc jobs

Used extensions & nodes

Created with KNIME Analytics Platform version 4.5.2
  • Go to item
    KNIME Base nodesTrusted extension

    KNIME AG, Zurich, Switzerland

    Versions 4.1.2, 4.5.2

    knime
  • Go to item
    KNIME Deep Learning - Keras IntegrationTrusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.1.0

    knime
  • Go to item
    KNIME ExpressionsTrusted extension

    KNIME AG, Zurich, Switzerland

    Versions 4.1.0, 4.5.1

    knime
  • Go to item
    KNIME Python Integration

    KNIME AG, Zurich, Switzerland

    Version 4.5.2

    knime
  • Go to item
    KNIME TextprocessingTrusted extension

    KNIME AG, Zurich, Switzerland

    Version 4.5.0

    knime

Legal

By using or downloading the workflow, you agree to our terms and conditions.

KNIME
Open for Innovation

KNIME AG
Talacker 50
8001 Zurich, Switzerland
  • Software
  • Getting started
  • Documentation
  • Courses + Certification
  • Solutions
  • KNIME Hub
  • KNIME Forum
  • Blog
  • Events
  • Partner
  • Developers
  • KNIME Home
  • Careers
  • Contact us
Download KNIME Analytics Platform Read more about KNIME Business Hub
© 2025 KNIME AG. All rights reserved.
  • Trademarks
  • Imprint
  • Privacy
  • Terms & Conditions
  • Data Processing Agreement
  • Credits