Skip to content

Developers

How can we help you?

← Go back

Introduction to Snitch

To track changes in any resource (title, metadata, etc.), you can use the Mediagenix On-Demand Events feed and Sequence Events feed. We often recommend you develop events-driven integrations that:

  • regularly poll the Mediagenix On-Demand Events feed and / or Sequence Events feed;
  • filtered by event-type to obtain only the events you’re interested in;
  • and using the parameter newer_than to obtain only the new events having occurred since your last request.

But in those cases when you need to build several integrations, performing several poll requests from multiple components may be inefficient. So we have developed a service that takes care of it for you, and we call this service Snitch.

Snitch delivers the kind of events that your integrations need to an Amazon SNS topic and/or Amazon FIFO SQS. Then your integrations can poll the FIFO SQS queue for messages. Or your integrations can either add a subscriber to the SNS topic in form of a standard Amazon SQS queue, AWS Lambda functions, or HTTP endpoints.

Configuration options

For each of your integrations, we can setup Snitch in three different ways depending on your needs and preferences:

  1. To publish to an Amazon SNS topic that you own. In this setup, you’ll configure and manage the SNS topic that Snitch publishes to, and this will give you total control over how you consume the events (e.g., email, SQS queues, Lambda functions…).

  2. To publish to an Amazon Standard SQS queue that we’ll configure for you. In this setup, you won’t care about configuring or managing SNS topics or SQS queues in AWS; you only need to consume the events from a Standard SQS queue using AWS APIs and SDKs.

  3. To publish to an Amazon FIFO SQS queue that we’ll configure for you. In this setup, you’ll get the same benefits as with the Standard SQS queue option with the additional benefit of preserved message order and deduplication. Preserved message order and deduplication are only provided as best-effort and not guaranteed by standard SQS queue.

How it works

During the initial setup of Snitch, you will specify which events (or all of them) from Mediagenix On-Demand and/or Sequence should be configured for delivery into an SNS topic and/or a FIFO SQS queue. Depending on your configuration, you’ll need to grant us publishing rights to your SNS topic, or we will provide you with a standard SQS queue endpoint or a FIFO SQS queue endpoint - and secure credentials - where your integration can start receiving messages from the queue as the default subscriber type. Please liase with your Mediagenix On-Demand Technical Account Manager (TAM) to have Snitch setup to your needs.

We recommend using the AWS SDK to consume messages from the queue. The SDK is available for multiple languages: Java, .NET, Node.js, Ruby, Python, Go, and more.

Payload

Each message that Snitch publishes corresponds to an Event that has happened in Mediagenix On-Demand or Sequence. The content of the messages is the same as a Mediagenix On-Demand API Event or Sequence API Event, with the difference that it is in JSON format.

{
  "id": 135023305,
  "event_type": "asset_metadata_updated",
  "timestamp": "2019-07-10T10:02:05.000+01:00",
  "subject": "https://movida.bebanjo.net/api/assets/2286764",
  "snitch_company_id": 28,
  "changes": "historical-catalogue-id",
  "subject_external_id": "rn2lrr1"
}
  • id: Event ID.
  • event_type: The type of the event.
  • timestamp: The date and time that the event occurred.
  • subject: The resource that was created, updated or deleted.
  • changes: List of attributes and links that changed; only present for _updated events.
  • subject_external_id: The external ID, if set, for the subject resource.

Message lifecycle in Standard and FIFO SQS queue

When using SQS to consume events, the messages follow a simple lifecycle to ensure that all messages are processed:

  • A new event happens in Mediagenix On-Demand or Sequence.
  • Snitch filters the events and publishes a new message for that event to the FIFO SQS queue, and/or SNS topic which by default has an SQS queue subscribed to it.
  • An integration performs a call to the queue, sees the pending message and retrieves it.
  • When the message is retrieved, the message becomes locked meaning that is not available for any other consumer that can read from the same queue.
  • The integration processes the message successfully and then deletes it from the queue, so no one else processes it. If this system fails to process the message, then it becomes available again as soon as the visibility timeout passes.

Message ordering in Standard SQS queue

Note that message ordering cannot be guaranteed due to the nature of Amazon Standard SQS. As Amazon notes, standard queues make a best effort to preserve the order of messages, but more than one copy of a message might be delivered out of order.

Standard queues provide a loose-FIFO (first in, first out) capability that attempts to preserve the order of messages. However, because standard queues are designed to be massively scalable using a highly distributed architecture, receiving messages in the exact order they are sent is not guaranteed.

Please reach out to your TAM if you have any questions!