• home
  • forum
  • my
  • kt
  • download
  • Amazon cloud computing SimpleDB boto processing

    Author: 2009-04-24 10:46:48 From:

    Amazon SDB is a fast, scalable real-time dataset indexing and querying framework that makes it easy to store and retrieve structured data for your Amazon Web Services-based applications. It's designed to work well with the other Amazon Web Services, such as Elastic Compute Cloud (EC2) and Simple Storage Service (S3). SDB enables you to build your entire application stack within the Amazon Web Services environment. You pay for the service based entirely upon your usage. There is also a free tier of service available.

    Some valuable features provided by SDB:

    Reliability
    SDB is designed to store your indexed data redundantly across multiple data centers and to make them available at all times.
    Speed
    SDB is designed to provide quick retrieval of data, especially if your requests are made from within the Amazon Web Services environment from an EC2 instance.
    Simplicity
    The programming model for accessing and using SDB is simple and can be used from a variety of programming languages.
    Security
    SDB is designed to provide a high level of security. Access to the data is restricted to authorized users.
    Flexibility
    SDB gives you the ability to store data on the fly without any need for pre-defined schemas.
    Inexpensive
    SDB charges are quite economical. You are only charged for what you actually use.

    This rest of this section explores the concepts that underpin SDB.

    Domains

    A domain is a container that lets you store your structured data and run queries against it. The data is stored in a domain as items. Conceptually, a domain is similar to a worksheet tab in a spreadsheet; items are rows in the spreadsheet. You can run queries against a domain, but you cannot yet query across domains in the current version of SDB.

    Each domain has the following metadata associated with it:

    • Date and time the metadata was last updated
    • Number of all items in the domain
    • Number of all attribute name-value pairs in the domain
    • Number of unique attribute names in the domain
    • Total size of all item names in the domain, in bytes
    • Total size of all attribute values, in bytes
    • Total size of all unique attribute names, in bytes

    SDB, like Simple Queue Service (SQS), follows the "eventual consistency" model. SDB maintains multiple copies of each domain for fault tolerance. Every change made to a domain is propagated across all copies.

    Amazon CTO Werner Vogels discusses the reasoning behind the concept of eventual consistency on his blog.

    Because this operation sometimes takes a few seconds, depending on system load and network latency, a consumer of your domain may not see the changes immediately. Changes will eventually be propagated throughout SDB, but this delay is an important consideration when designing your SDB-based applications.

    Items

    Items represent individual objects within your domains, and they contain attributes with values. Each item is conceptually similar to a row in a spreadsheet — an attribute is a column and the values are cells. Attributes are not restricted to single values and can even have multiple values. SDB automatically indexes your domains regardless of how the data is structured.

    SDB also has a time limit for executing any single query against your domains. If a query takes longer than 5 seconds, SDB will stop the query and return an error.

    Domains in SDB are flexible and don't have any fixed schemas. Each item within a domain can contain a unique set of up to 256 attributes. The attributes can even be completely different from all other attributes for the other items within that domain.

    Limitations

    The current version of SDB has limitations that you should consider when designing your application. Table 1 shows the limitations (as specified by Amazon in its latest documentation).


    Table 1. Current limitations

    ParameterCurrent restrictions
    Domain size 10 GB per domain
    250,000,000 attribute name-value pairs
    3-255 characters (a-z, A-Z, 0-9, '_', '-', and '.')
    Domains per Amazon Web Services account 100
    Attributes Name-value pairs per item is 256.
    Name length is 1024 bytes.
    Value length is 1024 bytes.

    Only allowed characters are UTF-8 characters that are valid in XML documents. Control characters and any sequences that are not valid in XML are not allowed.

    Per PutAttributes operation limited to 100
    Requested per Select or QueryWithAttributes operation limited to 256.
    Maximum items in query response 256
    Maximum query execution time 5 seconds
    Maximum predicates per query expression 10
    Maximum comparisons per query expression predicate 10
    Maximum number of unique attributes per select expression 20
    Maximum number of comparisons per select expression 20
    Maximum response size for QueryWithAttributes and Select 1 MB


    Pricing

    Amazon provides a free tier for SDB, along with pricing for usage above the free tier limit. The charges are based on:

    • The machine usage of each SDB request.
    • The amount of machine capacity used for completing the specified request, normalized to the hourly capacity of a 1.7-GHz Xeon processor.

    Free tier

    There are no charges on the first 25 machine hours, 1 GB of data transfer, and 1 GB of storage that you consume every month, at least until 1 Jun 2009. This is a significant amount of usage being provided for free for a limited time by Amazon. Many types of applications can operate very easily within this free tier. Table 2 shows example pricing.


    Table 2. Pricing for machine utilization

    QuantityCost
    First 25 machine hours Free
    Additional machine hours $0.14 per machine hour

    Table 3 addresses the amount of data transferred to and from SDB. There is no charge for data transferred between SDB and other Amazon Web Services within the same region. Data transferred between SDB and other Amazon Web Services across regions will be charged at Internet Data Transfer rates on both sides of the transfer.


    Table 3. Pricing for data transfer

    Type of transferCost
    All data transfer First 1 GB of data transfer in is free
    $0.100 per GB — all additional data transfer in

    First 1 GB of data transfer out is free
    $0.170 per GB — first 10 TB/month data transfer out
    $0.130 per GB — next 40 TB/month data transfer out
    $0.110 per GB — next 100 TB/month data transfer out
    $0.100 per GB — data transfer out / month over 150 TB

    Table 4 outlines costs for structured data storage.


    Table 4. Structured data storage

    Amount of storageCost
    All data storage First 1GB of data is free.
    $0.25 per GB /month - all additional data storage

    For the latest pricing, check Amazon SDB. You can also use the Simple Monthly Calculator provided by Amazon for calculating your monthly usage costs for SDB and the other Amazon Web Services.

    discuss this topic to forum

    relation tutorial

    No information

    Category

      Administration (17)
      Editing Files (5)
      Getting Started (13)
      Installation (10)
      Linux and other OSs (15)
      Miscellaneous (17)
      Networking (12)
      Security (13)
      Shells and Utilities (19)
      System Monitoring (8)
      Troubleshooting (1)
      X Windows (6)

    New

    Hot