Migrating from Scrypto v0.11.0 to v0.12.1

Product Versions

Engine, Node and Scrypto: [Compatibility: Scrypto 12.1]

App Building Tools: [Compatibility: Scrypto 12.1]

Public Applications: [Compatibility: Scrypto 12.1]

Changes from v12 to v12.1

This section captures changes between v12 and v12.1. The rest of the article captures changes between v11 and v12.

Please note that the "zabanet" network is currently running rcnet-v3 (compatible with Scrypto 12.0), so currently v12.1 can only be used locally, with resim. Zabanet (the "RCnet v3.x network") will be wiped and swapped to run rcnet-v3.1 (compatible with Scrypto 12.1) later this week.

Further Decimal Tweaks

In response to developer feedback, we have made the following changes:

  • Support for the arithmetic operators for Decimal and PreciseDecimal has been restored in Scrypto 12.1. Please be aware that these operations may panic, and be sure to read the overflow review recommendations in the below section.

  • All the safe_X methods have been renamed with checked_X, implementing the standard Rust checked traits from num_traits.

Please see the "Decimal Updates - Overview" section below for more information on the decimal changes.

Slight costing adjustments

Nothing too noticable. The cost for "state storage" has been reduced, but a few more elements are now covered.

Note that "state storage" is split into a cost for "state growth" (a cost for increasing the total size of the ledger state) and a cost for "transient data" needed to be saved by indexers, but not included in the main ledger state. The "transient" state cost essentially covers the raw transaction payload and the receipt (including things like events, logs, and state changes).

For the time being, the network charges for both "state growth" and "transient data" at a rate of 1MB ~ 6 USD worth of XRD, but we wish to mention that these numbers will likely diverge in future protocol updates, likely resulting in charging more for growing ledger state than for transient data.

What this means is, when optimizing for fees, you should aim to reduce both transient state output by your transaction (eg substate churn in your receipt) and also avoid unnecessary increases in the size of state stored on ledger.

URL relaxation

The URL metadata value validation now allows fragments in the URL. An example of a fragment is #my-title in ../index.html#my-title.

Improvements to native blueprint stub constructors

They now return a Global<X>, eg:

let account: Global<Account> = Blueprint::<Account>::create_advanced(..);

Receipt changes

The state changes on the transaction receipt have been adjusted slightly. If you need the old ones, simply do state_changes.into_legacy()

Type renames

  • The enum ObjectModuleId has been renamed to ModuleId (this enum represents "Main", or one of the native modules: "Metadata", "Royalty" or "RoleAssignment") .

  • The enum ModuleId has now been renamed to AttachedModuleId (this enum was just one of the native modules, but wasn’t exposed to Scrypto)

We have added a (deprecated) type alias type ObjectModuleId = ModuleId so that old code should still compile.

Updates to the manifest enums

More names are now supported, including Enum<ResourceOrNonFungible::Resource>(..) used in Access Rules and the Account allowed depositor features.

NonFungibleDataSchema and KeyValueStore changes

We have added a change behind the scenes which paves the way for large code size reductions for code utilizing KeyValueStore or NonFungibleData primitives. This code size reduction will be introduced in our upcoming final scrypto lib version before mainnet launch.

As part of this change, if you’re creating non fungible resources from the manifest, be aware that the the Tuple(Enum<0u8>(..)) wrapping the schema needs replacing with Enum<NonFungibleDataSchema::Local>(Enum<0u8>(..)).

Scrypto Updates (v11 - v12)

Decimal Updates - Overview

We welcome developer feedback on the #scrypto channel in Discord. In response to developer feedback, support for the arithmetic operators for Decimal and PreciseDecimal has been restored in Scrypto 12.1. Please be aware that these operations may panic, and be sure to read the overflow review recommendations in this section.

At some point after the Babylon upgrade, we’ll plan to revisit this issue to discuss further what options are available to draw attention to the possibility of overflow. We have captured the ideas already proposed, thanks to all who had suggestions.

One of the main changes this release is to our Decimal types and math. There are two parts:

  1. Decimal and PreciseDecimal have changed size.

    • Decimal was reduced from 256 bits to 192 bits, now with ~40 integral digits, and (still) exactly 18 fractional digits after the decimal point.

    • PreciseDecimal was reduced from 512 bits to 256 bits, and now has ~41 integral digits and exactly 36 fractional digits after the decimal point.

    • PreciseDecimal is now effectively no larger than Decimal, but it is more precise - and is intended to be used as an interim calculation tool, for eg multiplying two Decimals losslessly, or representing interim calculations with more precision.

    • These changes were made in response to feedback from the community, and partly as an evolution of Florian’s BalancedDecimal type. They were done to bring things better in line with real world scenarios, so that more DeFi cases could use the built-in types without paying excessive fees for unused digits.

    • As an example, multiplication and exponentiation with Decimals is now 25% and 40% faster in Scrypto, and PreciseDecimal is 66% faster for both.

  2. Mathematical operations have been revisited, to make it clearer which operations can panic.

    • Having reviewed various blueprints, we observed that operations were often performed in an order which could trigger overflow, and partly this was due to a lack of awareness.

    • These changes are primarily intended to draw attention to these operations for the developer, and allow them to handle them appropriately.

    • Methods which can panic now return an Option<_>, where None indicates some form of overflow.

    • For Scrypto 12.0, we have removed the trait impls (for eg *, - etc) and added safe_X methods for these operations (for different X).

    • Many mathematical operations have been replaced with a checked_ version which implements the associated "Checked" Rust trait. These methods do not panic on overflow/underflow, and allow you to gracefully handle these conditions.

    • As of Scrypto 12.1, support for arithmetic operators on Decimal/PreciseDecimal is restored. Please be aware that these operations may panic, and be sure to read the overflow review recommendations in this section.

Further changes include:

  • The large integer types BnumUxxx / BnumIxxx have been renamed to Uxxx / Ixxx

  • The relevant math operations between Decimal, PreciseDecimal and primitive types are now properly commutative (ie have the same resultant value and type, independent of whether you do A * B or B * A)

Decimal Updates - Specifics

The Decimal type now has the following characteristics:

  • Size: 192 bits

  • Fractional part: 18 digits (approx 60 bits)

  • Integer part: approx 40 digits (approx 132 bits)

  • Max: 3138550867693340381917894711603833208051.177722232017256447

  • Min: -3138550867693340381917894711603833208051.177722232017256448

The PreciseDecimal type now has the following characteristics:

  • Size: 256 bits

  • Fractional part: 36 digits (approx 120 bits)

  • Integer part: approx 41 digits (approx 136 bits)

  • max: 57896044618658097711785492504343953926634.992332820282019728792003956564819967

  • min: -57896044618658097711785492504343953926634.992332820282019728792003956564819968

Decimal Updates - Overflow review recommendations

We recommend reviewing any code which touches mathematical operations, and determining if it is structured correctly.

Some rules of thumb which might be helpful to avoid overflow include the following:

  • You should typically divide before multiplying. If necessary, to keep precision, you can:

    • Do calculations in sqrt space for increased precision and to avoid overflow.

    • Convert to a PreciseDecimal, eg with PreciseDecimal::from(decimal) and do calculations between two PreciseDecimal numbers with this extra fractional precision. Please note that PreciseDecimal is no longer singificantly larger than Decimal in the integral part of the number, so multiplying two Decimal numbers in PreciseDecimal space is no longer guaranteed to be overflow free. Either consider dividing first, or alternatively, implement your own arithmetic over the Decimal "atto" 10^(-18) subunits / PreciseDecimal 10^(-36) subunits using the fixed precision Ixxx / Uxxx types.

  • You should generally handle similar quantities together:

    • EG (a_amount / a_supply) * b_amount rather than (a_amount * b_amount) / a_supply

  • You should generally start by calculating known-fixed size or small quantities first, to avoid overflows during intermediate calculations:

    • EG (multiplier * (Decimal::ONE - proportion)) * b_total rather than (b_total * multiplier) - (b_total * multiplier * proportion)

  • For complex calculations, prefer published/peer-reviewed algorithms (subject to standard licensing / legal checks etc)

As part of this review, it’s good to ask "when could this overflow?", and "could an overflow here result in my application bricking?". An application is bricked if the component can no longer perform an action without erroring, and can result in funds becoming locked.

A hypothetical example is the following. Imagine you have a component which allows deposits and withdraw. Deposits are straight into a vault, but at withdraw time, an event gets emitted. As part of preparation of that event payload, a calculation is performed which (hypothetically) calculates the interim quantity prev_balance * prev_balance. This performs fine - until someone deposits a total of 10^21 tokens in the component, at which point this interim calculation now overflows on every call to withdraw, preventing the withdrawal of the funds.

Of course, this is not the only kind of issue that could cause a transaction to fail (and potentially brick a component). Other things to watch out for in a review are:

  • Storing unbounded state in (eg) a Vec or IndexMap in component state, rather than using a KeyValueStore.

  • Running unbounded calculations, causing the execution limit to be hit.

  • Producing unbounded amounts of data (eg emitting an event which exceeds the event payload size, or emitting an unbounded number of events).

  • Making external calls which panic, exceed the engine call stack depth limit for cross-component calls (currently 8 frames), or use too much fee (through execution, or state finalization overhead).

Decimal Updates - Performance optimization notes

There are also a couple of things to suggest, to optimize performance for mainnet launch:

  • The dec!("234.435") / pdec!("99.11") macros are currently doing a string parse at runtime, which is inefficient. We are planning to implement an improvement into scrypto lib to make this into a free-abstraction compile-time macro, but for now, if building a performance-critical application, prefer using constants such as Decimal::ONE or hardcoding your own const Decimals using Decimal(I192::from_digits([x, y, z])) which represents the number (x + y * 2^64 + z * 2^128) / 10^18.

  • If you are using a calculation space such as sqrt space, consider saving your constants (or even values) in that space, and inverting that in your front-end.

Renamed RADIX_TOKEN to XRD

The RADIX_TOKEN constant has been renamed to XRD

Added Royalty roles:

  • royalty_setter

  • royalty_setter_updater

  • royalty_locker

  • royalty_locker_updater

  • royalty_claimer

  • royalty_locker_claimer

Renamed RoundingMode variants

  • Added RoundingMode::MidpointNearestEven

Added WithdrawStrategy to more seamlessly handle resources with varying divisibility

The WithdrawStrategy either allows to specify that your withdraw should use the exact amount specified, or should be rounded in some manner to the resource’s divisibility.

  • Added take_advanced(amount: Decimal, strategy: WithdrawStrategy) to buckets and vaults

  • Added amount_for_withdrawal(amount: Decimal, strategy: WithdrawStrategy) to resource managers

  • Updated protected_withdraw to accept a WithdrawStrategy for pools

Update to the Pool blueprint

  • The instantiate function on all three pool blueprints now takes an additional OwnerRole argument. If you’re using pools in Scrypto then you will get a compile-time error with this update since the pool stubs have been updated to reflect this.

Update to Package Owner Badge

The non-fungible local id of auto-generated package owner badges (via the non-advanced PUBLISH_PACKAGE instruction) is now NonFungibleLocalId("[<hex encoded raw bytes of address>]"), instead of an RUID, consistently with Validators and Accounts.

{Fungible/NonFungible}{Bucket/Proof/Vault}

  • You can now use {Fungible/NonFungible}{Bucket/Proof/Vault} as function input or output, or inside blueprint struct.

  • Resource Manager in scrypto can now also burn FungibleBucket and NonFungibleBucket

Fix costing on large withdraws on non fungible vaults

  • Added an explicit limit parameter to NonFungibleVault::get_non_fungible_local_ids

Accounts support allowed depositors

Updates Account to add a set of allowed depositors that can deposit any resource regardless of the account’s resource preferences

Updates to try_deposit_ methods

They all take an extra parameter of an optional authorized depositor badge which is claimed to be both an authorized depositor for the account, and also on the worktop. See this PR for more details.

You will need to add None as an additional parameter to these method calls to pass no authorized depositor badge.

Renamed role assignment methods

  • Rename role assignment method set_role to set

  • Rename role assignment method set_owner_role to set_owner

  • Rename role assignment method lock_owner_role to lock_owner

Added get_reservation_address to Runtime to be able get the global address of a reservation:

  • Runtime::get_reservation_address(reservation: &GlobalAddressReservation) → GlobalAddress

fn my_func(reservation: GlobalAddressReservation) {
  let address = Runtime::get_reservation_address(&reservation);
  ...
}

Replaced Runtime::blueprint_id() with blueprint_id() on the Blueprint:

  • Replaced Runtime::blueprint_id() with <BlueprintName>::blueprint_id()

#[blueprint]
mod my_blueprint {
  struct MyBlueprint {}

  pub fn my_func() {
    let (reservation, address) =
                Runtime::allocate_component_address(MyBlueprint::blueprint_id());
    ...
  }
}

Add casting errors in Scrypto on invalid casts

Component casting will now throw an error on cast:

pub fn cast_to_validator(address: ComponentAddress) {
  // will panic if address is not a validator component
  let _validator: Global<Validator> = Global::from(address);
}

Metadata value - minor developer UX improvements

Conversions from ResourceAddress, ComponentAddress and PackageAddress to GlobalAddress have been added, allowing you to add metadata value of [resource_address_1, resource_address_2] which gets converted to a Vec<GlobalAddress>. Due to the limits of Rusts’s type system, if you want to mix and match different types, you still need to explicitly map each into a GlobalAddress, using eg GlobalAddress::from(address).

NonFungibleData derive

The NonFungibleData derive macro now works with generics, if you happen to want to use the same generic type for multiple different resources.

Bear in mind that each non-fungible resource still needs to specify a concrete type for its data, so any generic parameters will need filling as part of creating the non-fungible resource.

New Validator Component methods

The following methods were added to the Validator component: - total_stake_xrd_amount() - total_stake_unit_supply() - get_redemption_value(amount_of_stake_units) - accepts_delegated_stake()

Locked two-way metadata link validation between the validator and stake unit resource can be used to discover and verify that a given resource is correctly the stake unit resource of a validator.

  fn my_func(validator: Global<Validator>) {
    let result: bool = validator.accepts_delegated_stake();
    let result: Decimal = validator.total_stake_xrd_amount();
    let result: Decimal = validator.total_stake_unit_supply();
    let result: Decimal = validator.get_redemption_value(DECIMAL::ONE);
  }

AuthZone Blueprint

  • Added assert_access_rule to the AuthZone blueprint

Role Limits

  • Added role name length limit (100) and number of roles defined limit (50)

Runtime

  • Added Runtime::generate_ruid for Scrypto

  • Added Runtime::bech32_encode_address() to scrypto lib

You now have the ability to use a system call to bech32m encode addresses:

pub fn my_func(address: ComponentAddress) {
  let encoded: String = Runtime::bech32_encode_address(address);
  ...
 }

All URLs and Origins are now validated by the engine

They are validated according to the logic here for Origin and here for Url.

Note that URL fragments (ie URLs ending in something like #my-internal-link-name) are currently invalid - but this will be allowed for mainnet lanch.

Changes to metadata reading

The method get_metadata<T>(key: &str) now returns a Result<Option<T>, MetadataConversionError>, enabling handling of metadata which exists, but is of the wrong type.

Fixed missed return of role assignment methods

  • Updated AuthZone::pop to return an Option<Proof>

Various costing adjustments for RCNet v3

  • Updated the price of execution & finalization cost units

  • Updated USD/XRD rate and storage cost

  • Added archive storage cost (e.g. transaction payload)

  • Fixed a rounding bug in burning amount calculation

  • Removed fee burning for system transactions

Adjust mint limit to 2^152

This is part of the change to the Decimal size. Make sure to not mint or create resources with inital supply greater than 2^152.

Add Reservation Address to advanced constructors

  • Add optional AddressReservation to constructors of Account/AccessController/OneResourcePool/TwoResourcePool/MultiResourcePool

Costing Updates (v11 - v12)

Costs and Fees

Costs and fees have been adjusted to be inline with where we expect them to be for mainnet launch. Only small tweaks are expected.

Minimization built into scrypto build

Scrypto build now includes a code pruning / minimization step.

We are also continuing to work on (transparently) decreasing code size of scrypto lib, which will be included in the final scrypto lib release before mainnet launch in the coming weeks.

You can also improve your package size (and therefore fees) by using IndexMap/IndexSet (or even Vec) instead of HashMap/HashSet or BTreeMap/BTreeSet - because IndexMap/IndexSet can be included by the Scrypto lib already for some native calls, but the others are not. Although if you’re using a map or set to store state, you should also beware of storing unbounded state in your component - you probably want to be using a KeyValueStore in this case.

Transaction fee abstraction changes

Conceptually, the transaction fee is now comprised of:

  • Network execution (tip multiplier is applied)

    • This effectively tracks the time it takes to execute the transaction

    • Transactions which do lots of computation, load lots of state, or load lots of different packages will have a higher execution cost

  • Network finalization (tip multiplier is applied)

    • This effectively tracks the time it takes to process the transaction receipt and commit the transaction results

    • Transactions which update lots of substates, write lots of data, or emit lots of events or logs will have a higher finalization cost

  • Tip

    • This is a product of the tip multiplier on top of the sum of the execution and finalization costs

  • Storage Cost. This consists of:

    • State expansion cost (caused by increasing the size of the "current state" of the network)

    • Archival storage cost (caused by the need for archival services to store transient data, such as transaction payloads, and transaction receipts which include events, logs and substate updates/churn)

  • Royalties

Refactored TransactionReceipt to include finalization costs

The FeeSummary in the transaction receipt has been updated to make it easier to extract the key information about fees. It’s now split into:

  • CostingParameters

  • TransactionCostingParameters

  • TransactionFeeSummary

  • TransactionFeeDetails

  • FeeSource (Commit only)

  • FeeDestination (Commit only)

As part of this change, cost_unit_price and cost_unit_limit have been renamed to execution_cost_unit_price and execution_cost_unit_limit respectively, and finalization_cost_unit_price and finalization_cost_unit_limit have been added.

Testing Updates (v11 - v12)

Unit Testing - scrypto-test framework

Another major announcement is the new scrypto-test unit testing framework. Check out the full article on the scrypto-test framework here.

TestRunner changes

  • Create a TestRunner with the TestRunnerBuilder, eg: TestRunnerBuilder::new().without_trace().build();

  • If you’re storing TestRunner, the new type is now DefaultTestRunner

Integration Testing - ManifestBuilder improvements

  • All ManifestBuilder methods taking NonFungibleLocalId arguments now take an IntoIterator<Item = NonFungibleLocalId> - so that you can pass it in with an array [..] or btreeset!(..) - this is for consistency with the other methods.

  • New ManifestBuilder method: create_proof_from_account_of_non_fungible - which takes a single NonFungibleGlobalId

  • Both try_deposit_batch_or_abort and try_deposit_batch_or_refund now accept a batch. The old methods, hard-coding batch = ManifestExpression::EntireWorktop are now try_deposit_entire_worktop_or_abort / try_deposit_entire_worktop_or_refund

  • Added burn_non_fungibles_in_account / burn_non_fungible_in_account helper methods on manifest builder

  • Add call_metatadata_method etc

  • Renamed update_role to set_role for consistency, and changed its parameters to accept Into<X>

Fix suggestions:

  • If you have an error with a manifest builder method which used to take a &BTreeSet<NonFungibleLocalId>, just remove the reference &, or replace the BTreeSet with an array [..].

  • If you have an error with a manifest builder try_deposit_batch_or_X method, rename the method to try_deposit_entire_worktop_or_abort or try_deposit_entire_worktop_or_refund

  • If you have an error with a manifest builder update_role method, change the method name to set_role or set_main_role.

Integration Testing - TestRunner improvements

The first few are thanks to a137x in this PR:

  • Read decoded component struct: pub fn component_state()

  • Read decoded struct of NonFungible: pub fn get_non_fungible_data()

  • Read the value of a KeyValueStore entry: pub fn get_kv_store_entry()

  • Read all entries of a KeyValueStore, packed as a HashMap: pub fn get_all_kv_store_entries()

The following changes have also been made:

  • Added contains_non_fungible query to NonFungible vaults and buckets

  • Updated TestRunner inspect_non_fungible_vault to return iterator over non fungibles

  • Added execute_unsigned_built_manifest_with_faucet_lock_fee, execute_unsigned_built_manifest, execute_built_manifest_with_faucet_lock_fee and execute_built_manifest which all take a |builder| ⇒ { .. }. For signatures, the latter take anything which can be turned into signatures, including TransactionSignatures::None, as well as single/vecs/arrays of NonFungibleGlobalId, PublicKey and PublicKeyHash.

Scrypto-Unit updates

If you are storing the `test_runner`you will need to make the change shown below.

// Before
let mut test_runner = TestRunner::builder().build();
// Now
let mut test_runner = TestRunnerBuilder::new().build();

In addition you may find the need to update the following type TestRunner to be DefaultTestRunner.

Restored metadata display for resim

Metadata is now displayed for all relevant entities which are looked up in resim.

Manifest/Transaction Updates (v11 - v12)

Instruction changes

  • Rename CALL_ACCESS_RULES_METHOD to CALL_ROLE_ASSIGNMENT_METHOD

  • Add RECALL_NON_FUNGIBLES_FROM_VAULT manifest instruction

  • Add ASSERT_CONTAINS_ANY manifest instruction

  • Update CREATE_VALIDATOR to return bucket of change if too much fee is passed in

  • Add DROP_AUTH_ZONE_REGULAR_PROOFS

  • Add DROP_NAMED_PROOFS

  • Rename CLEAR_SIGNATURE_PROOFS to DROP_AUTH_ZONE_SIGNATURE_PROOFS

  • Rename CLEAR_AUTH_ZONE to DROP_AUTH_ZONE_PROOFS

Update to try_deposit_…​ account calls

  • Any try_deposit_…​ account calls should now have an extra None at the end, after the Bucket or batch of buckets, as part of the authorized depositor change.

CALL_METHOD
    Address("<ACCOUNT_ADDRESS>")
    "try_deposit_batch_or_abort"
    Expression("ENTIRE_WORKTOP")
    None
;

More specifically, the changes made are as follows:

  • The try_deposit_ functions have an additional argument at the end which gives the authorized depositor, and is of type Option<ResourceOrNonFungible>. This will normally be set to None unless you are using a depositor badge.

  • The try_deposit_batch_or_refund method now returns an Option<Vec<Bucket>>, and either all deposits succeed (returns None), or none succeed (returns Some containing all deposited buckets)

  • Split the configure_resource_deposit_rule method into two methods:

    • set_resource_preference: Sets the preference for a given resource. Overrides the preference if one already exists.

    • remove_resource_preference: Removes the preference for a given resource. No-OP if no resource presence exists for this resource.

  • In relation to the previous point, the ResourceDepositRule type has been renamed to ResourcePreference and the Neither variant has been dropped.

  • Renamed the change_account_default_deposit_rule method to set_default_deposit_rule.

  • In relation to the previous point, the AccountDefaultDepositRule type has been renamed to DefaultDepositRule.

For additional details see this PR.